Systems and methods for attack simulation on a production network

ABSTRACT

The disclosure is directed towards systems and methods for improving security in a computer network. The system can include a planner and a plurality of controllers. The controllers can be deployed within each zone of the production network. Each controller can be configured to assume the role of an attacker or a target for malicious network traffic. Simulations of malicious behavior can be performed by the controllers within the production network, and can therefore account for the complexities of the production network, such as stateful connections through switches, routers, and other intermediary devices. In some implementations, the planner can analyze data received from the controllers to provide a holistic analysis of the overall security posture of the production network.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/299,097, filed on Feb. 24, 2016 and entitled “SYSTEMS AND METHODSFOR ATTACK SIMULATION ON A PRODUCTION NETWORK,” which is incorporatedherein by reference in its entirety.

FIELD OF DISCLOSURE

The present disclosure is generally directed to various systems andmethods for configuring, executing and managing attack simulations in aproduction network environment.

BACKGROUND

Computer network security infrastructures can be complex. Typically,they include many different components that each play a role inpreventing, detecting, or responding to attacks and malicious behaviors.The security posture of an organization is the result of how theorganization's people, processes, and technology all respond to aspecific malicious behavior. Testing an organizations security postureto identify and correct security flaws can be complex and challenging.

BRIEF SUMMARY

Aspects and implementations of the present disclosure are generallydirected to systems and methods for improving security in a computernetwork. Organizations such as businesses often maintain large computernetworks for storing and accessing electronic information. A network caninclude many different types of computing devices, such as servers,desktop computers, laptop computers, mobile devices, switches, androuters that are connected to one another according to a networktopology. In some implementations, a network topology may be selected tofacilitate data transfer between devices in the network and also tosecure sensitive information from being accessed by unauthorized users.For example, a network may be divided into various zones, each of whichincludes computing devices of a particular type. In someimplementations, a group of servers may be organized into one zone of anetwork in which they are positioned in close physical proximity to oneanother and directly communicatively coupled to one another. Anothernetwork zone can include a group of client workstations that are used torequest access to data stored on the servers. In general, a network mayinclude any number of such zones.

Each zone of the network can be protected by various computer securitymechanisms. For example, a device such an intrusion prevention system(IPS), an intrusion detection system (IDS), or a firewall may bepositioned between communication paths connecting the various networkzones to one another. In addition, a series of routing and switchingdevices also may be included in the network to interconnect the variousnetwork zones, and also to interconnect computing devices within eachnetwork zone. As a result, the overall network topology may be verycomplex.

In some implementations, the security of each computing device in anetwork may be tested individually. For example, network packetsrepresenting malicious behavior may be directed towards one of thecomputing devices, and the computing device can be monitored todetermine whether it responds appropriately, such as by dropping themalicious packets or generating an alarm condition to indicate that thepackets may correspond to an attack. Typically, such a test may be runin a laboratory setting, to avoid compromising the computing deviceunder test in case the computing device does not successfully preventthe attack. However, such isolated lab testing scenarios may fail tofully validate the security posture of the more complex productionnetwork, even if individual computing devices appear to respondappropriately to malicious network traffic. For example, an attacker maybe able to take advantage of misconfigurations that exist in theproduction setup but are not present in the isolated laboratory testingscenario. Furthermore, laboratory testing typically relies on simplysending a stream of packets intended to replicate malicious behavior toa given computing device. As such, there is no way to test activestateful connections that may be necessary to route through in theproduction network environment. Therefore, isolated laboratory testingof computing devices cannot be used to determine how a complex networkwould respond to malicious packets.

Instead of performing isolated laboratory testing of individualcomputing devices as discussed above, security of a computer network canbe improved by testing within a system that is able to evaluate thesecurity posture of an organization's complex production network. Insome implementations, such a system can be included within a productionnetwork and configured such that the security posture of the productionnetwork can be evaluated without putting the computing devices withinthe production network at risk. For example, the system can include aplanner and a plurality of controllers. The controllers can be deployedwithin each zone of the production network. Each node can be configuredto assume the role of an attacker or a target for malicious networktraffic. Simulations of malicious behavior can be performed by thecontrollers within the production network, and can therefore account forthe complexities of the production network, such as stateful connectionsthrough switches, routers, and other intermediary devices. Moreover,simulated malicious network traffic can be constrained to take placeonly between controllers configured for this purpose, so that no clientsor servers of the production network are put at risk. The planner can beconfigured to communicate with each of the controllers to provideinstructions for carrying out simulations, and to receive data from thecontrollers corresponding to the results of the simulations. In someimplementations, the planner can analyze the data received from thecontrollers to provide a more complete analysis of the overall securityposture of the production network. The details of various embodiments ofthe present solution are set forth in the accompanying drawings and thedescription below.

Throughout this disclosure, the term “simulated attack” and “simulation”are used to indicate an attack that takes place between controllers in aproduction network, but which does not impact the functionality of theproduction equipment itself. However, it should be noted that in atleast some implementations, a simulated attack between controllers caninclude actual network traffic that is considered malicious. Forexample, in a simulated attack, the controllers may exchange actualmalware files or commands requesting the performance of actual maliciousactivity. Thus, it should be understood that the term “simulated attack”as used in this disclosure is intended to encompass behavior that isvirtually indistinguishable from an authentic attack, and may in someinstances be referred to as the controlled execution of an attack withina production network.

One inventive aspect of the subject matter of this disclosure can beimplemented in a method for controlling execution of malicious behaviorin a production network to test a security system of the productionnetwork. The method can include receiving, by a first controller on afirst node in a production network with a security system, instructionsto operate as an attacker and data for executing a predeterminedmalicious behavior on the production network. The method can includereceiving, by a second controller on a second node in the productionnetwork from the planner, instructions to operate as a target of thepredetermined malicious behavior by the attacker. The method can includeestablishing a connection between the first controller and the secondcontroller. The method can include transmitting, responsive to theinstructions by the first controller via the connection to the secondcontroller via at least a portion of the security system of theproduction network, network traffic comprising the predeterminedmalicious behavior and generated using the data. The method can includereceiving, by the first controller via the connection from the secondcontroller, one or more responses to the network traffic. The method caninclude determining, by the first controller, whether the one or moreresponses from the second controller are as expected.

In some implementations, the method can include receiving, by the firstcontroller, the instructions and data from a planner executing on atleast one node in the production network. In some implementations, themethod can include receiving, by the first controller, a plurality ofrequests for executing the predetermined malicious behavior via thenetwork traffic. In some implementations, the method can includereceiving, by the first controller, a plurality of responses to expectfrom the second controller.

In some implementations, the method can include receiving, by the secondcontroller, a plurality of responses to transmit responsive to acorresponding request from the first controller. In someimplementations, the method can include establishing the connection asat least one of a stateful connection or a secure connection. In someimplementations, the method can include transmitting, by the firstcontroller, network traffic comprises one or more requests to executethe malicious behavior.

In some implementations, the method can include generating, by thesecond controller, the one or more responses based at least on one ofinstructions or data received from a planner. In some implementations,the method can include comparing, by the first controller, the one ormore responses to expected response data received from a planner. Insome implementations, the method can include generating metadata aboutmismatches between the one or more responses and expected responses andcommunicating the metadata to a planner. In some implementations, thefirst controller and the second controller can maintain the networktraffic comprising the predetermined malicious behavior containedbetween the first controller and the second controller via theconnection.

Another inventive aspect of the subject matter of this disclosure can beimplemented in a system for controlling execution of malicious behaviorin a production network to test a security system of the productionnetwork. The system can include a first controller on a first node in aproduction network with a security system, configured to receiveinstructions to operate as an attacker and data for executing apredetermined malicious behavior on the production network. The systemcan include a second controller on a second node in the productionnetwork, configured to receive instructions to operate as a target ofthe predetermined malicious behavior by the attacker. The firstcontroller and the second controller can establish a connection.Responsive to the instructions, the first controller can be configuredto transmit, via the connection to the second controller via at leastportions of a security system of the production network, network trafficcomprising the predetermined malicious behavior and generated using thedata. The first controller can be configured to receive via theconnection from the second controller, one or more responses to thenetwork traffic, and to determine whether the one or more responses fromthe second controller are as expected.

In some implementations, the first controller is further configured toreceive the instructions and data from a planner executing on at leastone node in the production network. In some implementations, the firstcontroller is further configured to receive a plurality of requests forexecuting the predetermined malicious behavior via the network traffic.In some implementations, the first controller is further configured toreceive a plurality of responses to expect from the second controller.

In some implementations, the second controller is further configured toreceive a plurality of responses to transmit responsive to acorresponding request from the first controller. In someimplementations, the first controller and the second control are furtherconfigured to establish the connection as at least one of a statefulconnection or a secure connection. In some implementations, the firstcontroller is further configured to transmit the network trafficcomprising one or more requests to execute the malicious behavior.

In some implementations, the second controller is further configured totransmit the one or more responses based at least on one of instructionsor data received from a planner. In some implementations, the firstcontroller is further configured to compare the one or more responses toexpected response data received from a planner. In some implementations,the first controller is further configured to generate metadata aboutmismatches between the one or more responses and expected responses andcommunicate the metadata to a planner. In some implementations, thefirst controller and the second controller maintain the network trafficcomprising the predetermined malicious behavior contained between thefirst controller and the second controller via the connection.

Another inventive aspect of the subject matter of this disclosure can beimplemented in a method for configuring a controlled execution of asequence of attacks in a production network to test a security system.The method may include receiving, via a planner, a first specificationof a first attack of a plurality of attacks in an attack sequence. Thefirst specification may identify a first attack node and a first targetnode in a production network, a first network path selected between thefirst attack node and the first target node, and a first predeterminedmalicious behavior. A second specification of a second attack of theplurality of attack in the attack sequence may be received via theplanner. The second specification may identify a second attack node anda second target node in the production network, a second network pathselected between the second attack node and the second target node, anda second predetermined malicious behavior. Identification of one or moreconditions upon which to execute the second attack subsequent toexecution of the first attack in the attack sequence may be received viathe planner. The planner may store the attack sequence including thefirst specification of the first attack and the second specification ofthe second attack. The planner may identify the stored attack sequencefor controlled execution within the production network to test at leasta portion of a security system. In receiving the first specification, aselection of a pairing of the first attack node and the second attacknode may be received via interaction with a graphical representation ofa topology of the production network. In receiving the firstspecification, a selection of the first network path may be received viainteraction with a graphical representation of links between nodes inthe production network. The received second specification may identifythe second predetermined malicious behavior different than the firstpredetermined malicious behavior. The received second specification mayidentify the second attack node and the second target node includingtypes of operating systems different than at least one of the firstattack node or the first target node. The one or more conditions as atime duration between the first attack and the second attack may beidentified. The one or more conditions as detection of one of failure ofor success of the first attack may be identified.

Another inventive aspect of the subject matter of this disclosure can beimplemented in a system for configuring a controlled execution of asequence of attacks in a production network to test a security system.The system may include a planner executable on a processor coupled tomemory. The planner may be configured to receive a first specificationof a first attack of a plurality of attacks in an attack sequence, thefirst specification identifying a first attack node and a first targetnode in a production network, a first network path selected between thefirst attack node and the first target node, and a first predeterminedmalicious behavior, and receive a second specification of a secondattack of the plurality of attack in the attack sequence, the secondspecification identifying a second attack node and a second target nodein the production network, a second network path selected between thesecond attack node and the second target node, and a secondpredetermined malicious behavior. The planner may also be configured toreceive identification of one or more conditions upon which to executethe second attack subsequent to execution of the first attack in theattack sequence, store the attack sequence including the firstspecification of the first attack and the second specification of thesecond attack, and identify the stored attack sequence for controlledexecution within the production network to test at least a portion of asecurity system. The planner may be further configured to receive aselection of a pairing of the first attack node and the second attacknode via interaction with a graphical representation of a topology ofthe production network. The planner may be further configured to receivea selection of the first network path via interaction with a graphicalrepresentation of links between nodes in the production network. Theplanner may be further configured to receive the second specificationidentifying the second predetermined malicious behavior different thanthe first predetermined malicious behavior. The planner may be furtherconfigured to receive the second specification identifying the secondattack node and the second target node including types of operatingsystems different than at least one of the first attack node or thefirst target node. The planner may be further configured to receiveidentification of the one or more conditions as a time duration betweenthe first attack and the second attack. The planner may be furtherconfigured to receive identification of the one or more conditions asdetection of one of failure of or success of the first attack.

Another inventive aspect of the subject matter of this disclosure can beimplemented in a method for controlled execution of a sequence ofattacks in a production network to test at least a portion of a securitysystem of the production network. The method may include identifying, bya simulation data manager, an attack sequence including a first attackand a second attack and one or more conditions upon which to execute thesecond attack subsequent to the first attack. The first attack mayspecify a first attack node and a first target node in a productionnetwork, a first network path selected between the first attack node andthe first target node, and a first predetermined malicious behavior. Thesecond attack may specify a second attack node and a second target nodein the production network, a second network path selected between thesecond attack node and the second target node, and a secondpredetermined malicious behavior. The simulation data manager maycommunicate instructions to the first attack node and the first targetnode in the production network to initiate the first predeterminedmalicious behavior via the first path selected between the first attacknode and the first target node. A result data analyzer may monitorexecution of the first attack between the first attack node and thefirst target node for the one or more conditions. The simulation datamanager may determine, responsive to detecting the one or moreconditions via monitoring, to execute the second attack of the attacksequence. The simulation data manager may communicate instructions tothe second attack node and the second target node in the productionnetwork to initiate the second predetermined malicious behavior via thesecond path selected between the second attack node and the secondtarget node. The simulation data manager may identify the attacksequence including the first attack node and the first target node onnodes with a type of operating system different than at least one of thesecond attack node and second target node. The simulation data managermay identify the attack sequence including the first predeterminedmalicious behavior different than the second predetermined maliciousbehavior. The simulation data manager may identify the attack sequenceincluding the first selected network path different than the secondselected network path. Responsive to instructions from the planner via aconnection to the first target node and via at least a portion of thesecurity system of the production network, the first attack node maytransmit network traffic including the first predetermined maliciousbehavior. The one or more conditions including a time duration may bedetected via monitoring. The one or more conditions including at leastone of failure or success of the first predetermined malicious attackmay be detected via monitoring.

Another inventive aspect of the subject matter of this disclosure can beimplemented in a system for controlled execution of a sequence ofattacks in a production network to test at least a portion of a securitysystem of the production network. The system may include a simulationdata manager executable on a processor coupled to memory, and a resultdata analyzer. The simulation data manager may be configured to identifyan attack sequence including a first attack and a second attack and oneor more conditions upon which to execute the second attack subsequent tothe first attack. The first attack may specify a first attack node and afirst target node in a production network, a first network path selectedbetween the first attack node and the first target node, and a firstpredetermined malicious behavior. The second attack may specify a secondattack node and a second target node in the production network, a secondnetwork path selected between the second attack node and the secondtarget node, and a second predetermined malicious behavior. Thesimulation data manager may be configured to communicate instructions tothe first attack node and the first target node in the productionnetwork to initiate the first predetermined malicious behavior via thefirst path selected between the first attack node and the first targetnode. The result data analyzer may be configured to monitor execution ofthe first attack between the first attack node and the first target nodefor the one or more conditions. The simulation data manager, responsiveto the result data analyzer detecting the one or more conditions, may beconfigured to determine to execute the second attack of the attacksequence. The simulation data manager may be further configured tocommunicate instructions to the second attack node and the second targetnode in the production network to initiate the second predeterminedmalicious behavior via the second path selected between the secondattack node and the first second node. The simulation data manager maybe further configured to identify the attack sequence including thefirst attack node and the first target node on nodes with a type ofoperating system different than at least one of the second attack nodeand second target node. The simulation data manager may be furtherconfigured to identify the attack sequence including the firstpredetermined malicious behavior different than the second predeterminedmalicious behavior. The simulation data manager may be furtherconfigured to identify the attack sequence including the first selectednetwork path different than the second selected network path. The firstattack node may be configured to, responsive to instructions from aplanner, transmit via a connection to the first target node and via atleast a portion of the security system of the production network,network traffic including the first predetermined malicious behavior.The result data analyzer may be further configured to detect the one orconditions including a time duration. The result data analyzer may befurther configured to detect the one or conditions including at leastone of failure or success of the first predetermined malicious attack.

Another inventive aspect of the subject matter of this disclosure can beimplemented in a method for controlled execution of a malicious behaviorbetween multiple different components in a production network to test atleast a portion of a security system of the production network. Themethod can include identifying, by a server, a packet capture (PCAP)file to use for providing instructions for controlled execution ofmalicious behavior on a plurality of different components of aproduction network. The method can include identifying, by the server, afirst attack to execute against a first device of a plurality ofdifferent components, a second attack to execute against a second deviceof the plurality of different components, and a third attack to executeagainst a network component of the plurality of different componentsintermediary to the first end point device and the second device. Themethod can include communicating, by the server based on at least thePCAP file, a first set of instructions to a first pair of attack andtarget nodes in the production network to initiate the maliciousbehavior of the PCAP file against the first device, a second set ofinstructions to a second pair of attack and target nodes in theproduction network to initiate the malicious behavior of the PCAP fileagainst the second device, and a third set of instructions to a thirdpair of attack and target nodes in the production network to initiatethe malicious behavior of the PCAP file against the network componentintermediary to the first device and the second device. The method caninclude aggregating, by the server, results of each of the first attack,the second attack, and the third attack to provide an aggregated view ofthe malicious behavior between the first device and the second devicevia the network component.

In some implementations, one of the first device or the second device isan end point device. In some implementations, the method can includedetermining, by the server, from processing of the PCAP file one of atype of application traffic or communication protocol represented by thePCAP file. In some implementations, the method can include extracting,by the server, from the PCAP file an application layer record ofrequests and responses exchanged between an attacker and a targetcaptured in the PCAP file. In some implementations, the method caninclude identifying, by the server, the malicious behavior from the PCAPfile. In some implementations, the method can include extracting, by theserver, from the PCAP file content of a malware file for communicatingthe malicious behavior.

In some implementations, the method can include identifying, by theserver, a first network path selected for the first attack, a secondnetwork path selected for the second attack and a third network pathselected for the third attack. In some implementations, the method caninclude communicating, by the server, to each of the first pair ofattack and target nodes, second pair of attack and target nodes andthird part of attack and target nodes instructions comprising metadatadetermined from the PCAP file. In some implementations, the method caninclude receiving, by the server, results of controlled execution ofeach of the first attack between the first attack node and first targetnode of the first attack-target node pair, the second attack between thesecond attack node and second target node of the second attack-targetnode pair, and the third attack between the third attack node and thirdtarget node of the third attack-target node pair. In someimplementations, the method can include communicating, by the server,the aggregated results to an event management device to determinesecurity events that occurred during the controlled execution of atleast one of the first attack, the second attack, or the third attack.

Another inventive aspect of the subject matter of this disclosure can beimplemented in a system for controlled execution of a malicious behaviorbetween multiple different components in a production network to test atleast a portion of a security system of the production network. Thesystem can include a server. The server can be configured to identify apacket capture (PCAP) file to use for providing instructions forcontrolled execution of malicious behavior on a plurality of differentcomponents of a production network. The server also can be configured toidentify a first attack to execute against a first device of a pluralityof different components, a second attack to execute against a seconddevice of the plurality of different components, and a third attack toexecute against a network component of the plurality of differentcomponents intermediary to the first end point device and the seconddevice. The server can be configured, based on at least the PCAP file,to communicate a first set of instructions to a first pair of attack andtarget nodes in the production network to initiate the maliciousbehavior of the PCAP file against the first device, a second set ofinstructions to a second pair of attack and target nodes in theproduction network to initiate the malicious behavior of the PCAP fileagainst the second device, and a third set of instructions to a thirdpair of attack and target nodes in the production network to initiatethe malicious behavior of the PCAP file against the network componentintermediary to the first device and the second device. The server alsocan be configured to aggregate results of each of the first attack, thesecond attack, and the third attack to provide an aggregated view of themalicious behavior between the first device and the second device viathe network component.

In some implementations, one of the first device or the second device isan end point device. In some implementations, the server is furtherconfigured to determine from processing of the PCAP file one of a typeof application traffic or communication protocol represented by the PCAPfile. In some implementations, the server is further configured toextract from the PCAP file an application layer record of requests andresponses exchanged between an attacker and a target captured in thePCAP file. In some implementations, the server is further configured toidentify the malicious behavior from the PCAP file.

In some implementations, the server is further configured to extractfrom the PCAP file content of a malware file for communicating themalicious behavior. In some implementations, the server is furtherconfigured to identify a first network path selected for the firstattack, a second network path selected for the second attack and a thirdnetwork path for the third attack.

In some implementations, the server is further configured to communicateeach of the first pair of attack and target nodes, second pair of attackand target nodes and third part of attack and target nodes instructionscomprising metadata determined from the PCAP file.

In some implementations, the server is further configured to receiveresults of controlled execution of each of the first attack between thefirst attack node and first target node of the first attack-target nodepair, the second attack between the second attack node and second targetnode of the second attack-target node pair, and the third attack betweenthe third attack node and third target node of the third attack-targetnode pair. In some implementations, the server is further configured tocommunicate the aggregated results to an event management device todetermine security events that occurred during the controlled executionof at least one of the first attack, the second attack or the thirdattack.

BRIEF DESCRIPTION OF THE FIGURES

The foregoing and other objects, aspects, features, and advantages ofthe disclosure will become more apparent and better understood byreferring to the following description taken in conjunction with theaccompanying drawings, in which:

FIG. 1A shows a block diagram of an example system for improving networksecurity;

FIG. 1B shows a block diagram of an example planner included in thesystem network of FIG. 1A;

FIG. 1C shows a block diagram of an example controller included in thesystem network of FIG. 1A;

FIG. 2 is a flowchart of an example method for evaluating security in acomputer network;

FIG. 3 is an example graphical user interface for displaying a networktopology;

FIG. 4 is an example graphical user interface for selecting a particularmalicious behavior to be simulated in a computer network;

FIG. 5 is an example graphical user interface for displaying an attacksequence and an associated path through a computer network;

FIG. 6 is a first example graphical user interface for displayingresults of an attack sequence simulation in a computer network;

FIG. 7 is a second example graphical user interface for displayingresults of an attack sequence simulation in a computer network;

FIG. 8 is a third example graphical user interface for displayingresults of an attack sequence simulation in a computer network;

FIG. 9 is a fourth example graphical user interface for displayingresults of an attack sequence simulation in a computer network;

FIG. 10 is a flowchart of an example method for controlling execution ofmalicious behavior in a production network to test a security system ofthe production network;

FIG. 11A is a flowchart of an example method for configuring acontrolled execution of a sequence of attacks in a production network totest a security system;

FIG. 11B is a flowchart of an example method for controlled execution ofa sequence of attacks in a production network to test at least a portionof a security system of the production network;

FIG. 12 is a flowchart of an example method for controlled execution ofa malicious behavior between multiple different components in aproduction network to test at least a portion of a security system ofthe production network; and

FIGS. 13A and 13B are block diagrams of implementations of an examplecomputing device.

The features and advantages of the present invention will become moreapparent from the detailed description set forth below when taken inconjunction with the drawings, in which like reference charactersidentify corresponding elements throughout. In the drawings, likereference numbers generally indicate identical, functionally similar,and/or structurally similar elements.

DETAILED DESCRIPTION

For purposes of reading the description of the various embodimentsbelow, the following descriptions of the sections of the specificationand their respective contents may be helpful:

Section A describes a network environment and computing environmentwhich may be useful for practicing embodiments described herein;

Section B describes graphical user interfaces (GUIs) which may be usefulfor practicing embodiments described herein;

Section C describes techniques for controller-to-controllercommunication which may be useful for testing security of a productionnetwork;

Section D describes attack simulation sequences which may be useful fortesting security of a production network;

Section E describes packet capture (PCAP) traffic recording techniqueswhich may be useful for testing security of a production network; and

Section F describes architectures for computing devices which may beuseful for implementing the network environment and computingenvironment described in Section A.

A. Network and Computing Environment

Aspects and implementations of the present disclosure are directed tosystems and methods for improving security in a computer network.Organizations such as businesses often maintain large computer networksfor storing and accessing electronic information. A network can includemany different types of computing devices, such as servers, desktopcomputers, laptop computers, mobile devices, switches, and routers thatare connected to one another according to a network topology. In someimplementations, a network topology may be selected to facilitate datatransfer between devices in the network and also to secure sensitiveinformation from being accessed by unauthorized users. For example, anetwork may be divided into various zones, each of which includescomputing devices of a particular type. In some implementations, a groupof servers may be organized into one zone of a network in which they arepositioned in close physical proximity to one another and directlycommunicatively coupled to one another. Another network zone can includea group of client workstations that are used to request access to datastored on the servers. In general, a network may include any number ofsuch zones.

Each zone of the network can be protected by various computer securitymechanisms. For example, a device such an intrusion prevention system(IPS), an intrusion detection system (IDS), or a firewall may bepositioned between communication paths connecting the various networkzones to one another. In addition, a series of routing and switchingdevices also may be included in the network to interconnect the variousnetwork zones, and also to interconnect computing devices within eachnetwork zone. As a result, the overall network topology may be verycomplex.

In some implementations, the security of each computing device in anetwork may be tested individually. For example, network packetsrepresenting malicious behavior may be directed towards one of thecomputing devices, and the computing device can be monitored todetermine whether it responds appropriately, such as by dropping themalicious packets or generating an alarm condition to indicate that thepackets may correspond to an attack. Typically, such a test may be runin a laboratory setting, to avoid compromising the computing deviceunder test in case the computing device does not successfully preventthe attack. However, such isolated lab testing scenarios may fail tofully validate the security posture of the more complex productionnetwork, even if individual computing devices appear to respondappropriately to malicious network traffic. For example, an attacker maybe able to take advantage of misconfigurations that exist in theproduction setup but are not present in the isolated laboratory testingscenario. Furthermore, laboratory testing typically relies on simplysending a stream of packets intended to replicate malicious behavior toa given computing device. As such, there is no way to test activestateful connections that may be necessary to route through in theproduction network environment. Therefore, isolated laboratory testingof computing devices cannot be used to determine how a complex networkwould respond to malicious packets.

Instead of performing isolated laboratory testing of individualcomputing devices as discussed above, security of a computer network canbe improved by testing within a system that is able to evaluate thesecurity posture of an organization's complex production network. Insome implementations, such a system can be included within a productionnetwork and configured such that the security posture of the productionnetwork can be evaluated without putting the computing devices withinthe production network at risk. For example, the system can include aplanner and a plurality of controllers. The controllers can be deployedwithin each zone of the production network. Each node can be configuredto assume the role of an attacker or a target for malicious networktraffic. Simulations of malicious behavior can be performed by thecontrollers within the production network, and can therefore account forthe complexities of the production network, such as stateful connectionsthrough switches, routers, and other intermediary devices. Moreover,simulated malicious network traffic can be constrained to take placeonly between controllers configured for this purpose, so that no clientsor servers of the production network are put at risk. The planner can beconfigured to communicate with each of the controllers to provideinstructions for carrying out simulations, and to receive data from thecontrollers corresponding to the results of the simulations. In someimplementations, the planner can analyze the data received from thecontrollers to provide a more complete analysis of the overall securityposture of the production network.

Throughout this disclosure, the terms “simulated attack” and“simulation” are used to indicate an attack that takes place betweencontrollers in a production network, but which does not impact thefunctionality of the production equipment itself. However, it should benoted that in at least some implementations, a simulated attack betweencontrollers can include actual network traffic that is consideredmalicious. For example, in a simulated attack, the controllers mayexchange actual malware files or commands requesting the performance ofactual malicious activity. Thus, it should be understood that the term“simulated attack” as used in this disclosure is intended to encompassbehavior that is virtually indistinguishable from an authentic attack,and may in some instances be referred to as the controlled execution ofan attack within a production network.

FIG. 1A shows a block diagram of an example system 100 for improvingnetwork security. The system 100 includes an internal enterprise network105, shown within broken lines, as well as an external cloud network110. In some implementations, the enterprise network 105 can be anetwork maintained by an organization, such as a business or governmententity, for storing and accessing electronic information and forfacilitating electronic communication among members of the organization.The cloud network includes a planner 115 and a controller 140 e. Theenterprise network 105 includes several computing devices grouped intoseveral different zones. For example, a “workstations” zone includesseveral workstations 120 a-120 c, a “wireless” zone includes severalwireless devices 125 a-125 c, an “internal servers” zone includesseveral servers 130 a-130 c, and a “demilitarized zone” (also referredto as a “DMZ zone”) that includes several computing devices 135-135 c.Each of the network zones in the enterprise network 105 also includes arespective one of the controllers 140 a-140 d. The computing devices inthe various zones of the enterprise network 105 are communicativelycoupled to one another, and to the external cloud network 110, through arouter 145. Various network security mechanisms are positioned along thecommunication paths between the router 145, the network zones of theenterprise network 105, and the external cloud network 110. For example,a firewall 150 a is positioned between the router 145 and the cloudnetwork 110, an intrusion protection system (IPS) 155 d and a firewall150 b are positioned between the router 145 and the DMZ zone, an IPS 150c is positioned between the router 145 and the internal servers zone, anIPS 155 b is positioned between the router 145 and the wireless zone,and an IPS 155 a is positioned between the router 145 and theworkstations zone. Finally, a security information and event management(SIEM) device 160 is included within the enterprise network 105 andcommunicatively coupled to the cloud network 110. The SIEM device 160also can be communicatively coupled to each of the firewalls 150 a and150 b and to each of the IPSs 115 a-155 d.

The system 100 can be configured to facilitate comprehensive evaluationof the security posture of the enterprise network 105. In someimplementations, testing of the security posture of the enterprisenetwork 105 can be carried out by the planner 115 and the controllers140. Because the controllers 140 are positioned within the enterprisenetwork 105, the tests performed by the controllers 140 can accuratelyreflect the complexity of the enterprise network 105 better thanisolated laboratory testing of individual components taken out of thenetwork topology. Furthermore, because tests do not rely oncommunications between the production computing devices in each networkzone (i.e., the computing devices 135 in the DMZ zone, the servers 130in the internal server zone, the wireless devices 125 in the wirelesszone, and the workstations 120 in the workstation zone), the productioncomputing devices are never put at risk during testing.

In some implementations, the controllers 140 serve only to test thesecurity of the enterprise network 105. That is, the controllers 140 maynot be used for typical day-to-day operations of the enterprise network105. For example, although the controller 140 d is positioned within theworkstation zone alone with the workstations 120 a-120 d, the controller140 d itself may not serve as a workstation. Rather, the controller 140d is positioned within the workstation group for the purpose of testinghow a workstation 120 would respond to malicious behavior withoutactually putting any of the workstations 120 at risk. Likewise, thecontroller 140 c is positioned within the wireless group, but may not bea production wireless device like the wireless devices 125, thecontroller 140 b may not be an internal server like the internal servers130, and the controller 140 a may not be a computing device that servesthe same purpose as the computing devices 135. Instead, the controllers140 a-140 d serve only to test the expected response behavior of devicesin their respective zones when exposed to malicious network traffic. Thecontrollers 140 can assume the roles of attacker and target during asimulation of malicious behavior, so that the security posture of theenterprise network 105 can be evaluated without putting the otherdevices in the enterprise network 105 at risk.

In some implementations, the planner 115 can manage the controllers 140in order to facilitate the execution of simulated attacks by thecontrollers 140. For example, the planner 115 can send to thecontrollers 140 instructions regarding specific simulated maliciousbehaviors that the controllers 140 are to execute. The planner 115 alsocan receive from the controllers 140 information corresponding to theresults of simulations. Generally, management data between the planner115 and the controllers 140 can include initial registration of each ofthe controllers 140 with the planner 115, configuration instructionssent to the controllers 140, simulation execution instructions sent tothe controllers 140, status updates, and simulation result data.Management data is discussed in greater detail below. In someimplementations, the planner 115 can communicate with each of thecontrollers 140 via an encrypted channel. Each controller 140 caninclude a dedicated channel (e.g., a dedicated IP interface) to theplanner 115.

In general, a simulation can include data exchanged between twocontrollers 140 that is intended to replicate the network traffic thatwould be seen in the event of an attack or other form of maliciousbehavior. Because there are controllers 140 positioned within each zoneof the enterprise network 105, the traffic patterns (i.e., requests andresponses between controllers 140) of such simulations can be identicalto the traffic patterns that would be observed during an actual attack.In some implementations, the data exchanged between two controllers 140during a simulation can emulate data that would be sent during an actualattack. In one example, the planner 115 can send instructions to two ofthe controllers, such as the controller 140 a in the DMZ zone and thecontroller 140 d in the workstations zone, to execute a particularsimulation. The planner can instruct either of the two controllers 140 aand 140 d to act as the attacker, and the other to act as the target.Each controller 140 a and 140 d can receive from the planner 115information corresponding to the network traffic it should send to theother controller 140 a or 140 d, as well as information corresponding tothe responses it should expect from the other controller 140 a or 140 d.The two controllers 140 a and 140 d can then create connections betweenone another, which may be stateful connections through the router 145,the IPSs 155 a and 155 d, and the firewall 150 b. Simulation data canthen be transmitted between the controllers 140 a and 140 d according tothe simulation instructions they received from the planner 115. Duringthe simulation, each of the controllers 140 a and 140 d can compare theresponses it receives to the responses it expects to receive. Forexample, in some implementations, the expected response may differ fromthe actual response in implementations in which the IPSs 155 a and 155 dor the firewall 150 b drops packets after determining that the networktraffic associated with the simulation may be malicious. In someimplementations, the controllers 140 a and 140 d may then report theresults of the simulation to the planner 115. In some implementations,the security mechanisms within the enterprise network 105, such as theIPSs 155 a and 155 d or the firewall 150 b, can report information tothe SIEM device 160, which in turn can report this information to theplanner 115. The planner 115 can then aggregate and analyze the data itreceives from the controllers 140 a and 140 d and the SIEM device 160 todetermine whether the simulated attack was successfully prevented ordetected by the IPSs 155 a and 155 d or the firewall 150 b. Additionaldetails relating to simulated attacks are discussed further below inconnection with FIG. 2.

It should be understood that the above description presents only asingle non-limiting example of a simulated attack, and many othersimulations may be run in a similar manner. For example, in someimplementations, any one of the controllers 140 a-140 e can act as atarget, and any other of the controllers 140 a-140 e can act as theattacker for another simulation. Thus, because the system 100 includesthe controller 140 e that is external to the enterprise network 110,simulations can be run to emulate an attack originating from outside ofthe enterprise network 110, for example, by designating the controller140 e to serve as the attacker during a simulation. The path through theenterprise network (and therefore the security mechanisms in theenterprise network) that are tested can be determined based on thetopology of the enterprise network 105 and the location of the attackerand target controllers 140. In some implementations, the planner 115 canstore information about the topology of the enterprise network 105,including information corresponding to which controllers 140 arereachable from other controllers 140. The planner 115 can use thisinformation to ensure that simulations are only performed betweencontrollers that are positioned within zones that are actually reachablefrom one another, so that time and resources are not spent on simulatingattacks that would not be possible to carry out.

It should be understood that, while the planner 115 is shown as locatedwithin the external cloud network 115, in some implementations theplanner 115 can instead be located within the enterprise network 105.Furthermore, while the planner 115 and the controller 140 e are shown astwo separate entities within the cloud network 110, in someimplementations a single computing device may be used to implement thefunctionality of both the planner 115 and the controller 140 e. Inimplementations in which the planner 115 is instead located within theenterprise network 110, the functionality of the planner 115 and atleast one of the controllers 140 a-140 d in the enterprise network maybe implemented by a single computing device. For example, the planner115 can be positioned within the DMZ zone, and the functionality of theplanner 115 and the controller 140 a within that zone can be implementedby a single computing device.

FIG. 1B shows a block diagram of an example planner 115 included in theexample system of FIG. 1A. The planner 115 includes a simulation datamanager 116, a result data manager 117, and a database 118. Also shownin FIG. 1B is a plurality of packet capture (PCAP) files 119, which canbe received by the planner 115.

As discussed above, the planner 115 can serve to manage the execution ofsimulated attacks that take place between the controllers 140 shown inFIG. 1A. As shown in FIG. 1A, the planner 115 can be a centralizedcomputing device that exists in an external cloud network, such as theinternet. In some other implementations, the planner 115 can be deployedwithin the enterprise network 105 and maintained by the organizationthat maintains the enterprise network 105.

The simulation data manager 116 can be configured to handle thetransmission and receipt of data related to attack simulations. Forexample, in some implementations, the simulation data manager 116 cansend and receive information related to initial registration of eachcontroller 140, configuration of each controller 140, simulationexecution instructions, and simulation results. Initial registration canbe any process that pairs each of the controllers 140 with the planner115. In some implementations, a private key exchange can take placebetween each controller 140 and the planner 115, which can help toensure that no other computing devices can impersonate one of thecontrollers 140, which could compromise the security of the enterprisenetwork 105 in which the controllers 140 are deployed.

The simulation data manager 116 also can generate and transmit to thecontrollers 140 simulation execution instructions. Generally, simulationexecution instructions include any information necessary for thecontrollers 140 to carry out a simulated attack. For example, simulationinstructions can include a collection of all of the requests that eachcontroller 140 should make during a simulation, as well as all of theresponses that the each controller 140 should receive during thesimulation. Simulation execution instructions can be based on an actualattack that has been carried out in the past and that could potentiallypose a threat to the enterprise network 105. In some implementations,the simulation data manager 116 can generate instructions for such asimulation based on PCAP files corresponding to the actual attack. Forexample, the PCAP files 119 can be received by the planner 115 andstored in the database 118. The simulation data manager 116 can retrievea PCAP file 119 from the database 118 and can process the PCAP file 119to generate simulation instructions that will cause the controllers 140to send network traffic to replicate the attack. In someimplementations, the simulation data manager 116 can analyze the PCAPfile 119 to extract the application-layer record of requests andresponses exchanged between the attacker and the target. In one example,a PCAP file 119 may correspond to an attack in which an attackerreceives an HTTP GET request from a target, and subsequently redirectsthe request to an end server which causes a piece of malware to beinstalled on the target device. The PCAP file 119 will typically containlow-level packet information related to data exchanged during theattack. However, the simulation data manager 116 can process the PCAPfile 119 to extract the higher-level application layer record of eachHTTP request and response, including the content of the actual malwarefile, to create a set of instructions for the controllers 140 so thatthe controllers 140 can accurately replicate the attack within theenterprise network 105.

In some implementations, the simulation data manager 116 can beconfigured to process a PCAP file by first identifying each hostconversation within the PCAP file. For example, the simulation datamanager 116 can make this determination based on the communicationprotocol exhibited in the PCAP file. The simulation data manager 116also can determine the type of application traffic represented by thePCAP file, such as HTTP traffic that may be sent using TCP, or DNStraffic that may be sent using UDP. In some implementations, thedetermination of the type of application traffic represented by the PCAPfile can be made based on the use of application signatures in the PCAPfile.

The simulation data manager 116 can be configured to extract higherlevel (e.g., application level) data from each conversation present inthe PCAP file. In some implementations, this extraction can be performedbased on the type of application data within each conversation. Thesimulation data manager 116 can save the higher level data in thedatabase 118. In some implementations, the simulation data manager 116also can generate metadata to be included in the simulationinstructions. Metadata generated by the simulation data manager 116 canbe associated with the higher level data. For example, metadata caninclude the communication protocols in use and, in instances in whichthe protocol is TCP or UDP, a list of the significant ports used duringthe conversation. The simulation data manager 116 can then assign rolesto each higher level request and response extracted from the PCAP filesbased on the type of application that created the original conversationrepresented in the PCAP file. In some implementations, the rolesassigned by the simulation data manager 116 may be generic roles such as“attacker” and “target,” or “client” and “server.” In some otherimplementations, the roles assigned by the simulation data manager 116may be more specific, such as “DNS server.” Metadata provided to thecontrollers 140 can help the controllers 140 to determine whether theresponses they receive are similar to the expected responses, so thatthe controllers 140 can determine whether the attack was successfullyreplicated or whether security mechanisms within the network 105intervened to stop the attack. Metadata also can provide additionaldetails on how each controller should make the requests associated withthe simulation. In some implementations, the simulation data manager 116can generate metadata including how many separate sockets or connectionsshould be used when simulating an attack, which IP protocols andsignificant ports should be used, what type of application data shouldbe used in each connection, whether the data is a valid representationof the leveraged application or protocol, and what role each controller140 should serve in the simulation (i.e., whether each controller 140should act as an attacker or a target).

The simulation data manager 116 can manage simulations executed by thecontrollers 140 in a variety of ways. In some implementations, thesimulation data manager 116 can communicate directly with each of thetwo controllers 140 that are to be involved in a simulation (i.e., afirst controller 140 that will act as an attacker, and a secondcontroller 140 that will act as a target). In such implementations, thesimulation data manager 116 can send simulation data directly to both ofthe first and second controllers 140. The simulation data sent to thecontrollers 140 can include any of the data discussed above, includinginformation corresponding to the requests that each controller 140should send and metadata corresponding to the responses that eachcontroller 140 should expect to receive. In response to sending themanagement data, the simulation data manager 116 can receive anacknowledgement from each of the controllers 140 indicating that theyhave received the management data and are ready to begin executing thesimulation. The simulation data manager 116 can then send additionalmanagement data to the first controller 140 (i.e., the attacker)instructing the first controller 140 to begin the simulation. In someimplementations, the simulation data manager 116 can continue to sendand receive information to and from the first and second controllers140. For example, the simulation data manager 116 may send statusrequests to each of the first and second controllers 140 to determinewhether the simulation has completed. After the simulation data manager116 receives responses from both of the controllers 140 indicating thatthe simulation has completed, the simulation data manager 116 canreceive information corresponding to the results of the simulation fromeach of the first and second controllers 140. In some implementations,the simulation data manager 116 can be configured to “pull” the resultinformation from the first and second controllers 140 by explicitlyrequesting that the controllers 140 send the result information. In someother implementations, the controllers 140 can be configured to “push”their respective result data to the simulation data manager 116 of theplanner 115, and the simulation data manager 116 can receive the resultinformation from each of the controllers 140 without first requestingthe result information.

In some implementations, the simulation data manager 116 may interactwith only one of the controllers 140 involved in a simulation. Forexample, all of the data for the simulation, including the data that asecond controller 140 will require in order to execute the simulation,can be sent by the simulation data manager 116 to the first controller140. The first controller 140 can then send management data to thesecond controller 140, and the simulation can be executed. After thesimulation is complete, the second controller 140 can send its resultdata directly to the first controller 140 instead of to the simulationdata manager 116. The simulation data manager 116 can then receive theaggregated simulation results from both controllers 140 from the firstcontroller 140.

The result data analyzer 117 can be configured to generate informationrelated to the results of one or more simulations. In someimplementations, the information generated by the result data analyzer117 can be based on information received from the controller 140 andfrom the SIEM device 160. For example, the result data analyzer 117 canbe configured to query the SIEM device 160 for information related tosecurity events that may have occurred in the enterprise network 105during a simulation. As discussed above, the SIEM device 160 can beconfigured to receive such information from the various securitymechanisms in the enterprise network 105, such as the firewalls 150 aand 150 b, and the IPSs 155 a-155 d. All of the information received bythe SIEM device 160 can be sent to the result data analyzer 117 of theplanner 115. In some implementations, the database 118 may containinstructions to be used by the result data analyzer 117 for querying theSIEM device 160. For example, the SIEM device 160 may be a commercialproduct developed by a third party. The database 118 can containapplication programming interfaces (APIs) associated with various thirdparty SIEM devices 160, and the result data analyzer 117 can beconfigured to retrieve the appropriate API from the database 118 and usethe API to send and receive information with the SIEM device 160.

In some implementations, the result data analyzer 117 also can receiveinformation from each of the controllers 140 after a simulation hascompleted. For example, the result data analyzer 117 can receivemetadata generated by the controllers 140 during execution of asimulation. Such metadata may include the start time and end time forthe simulation, the sockets and ports used during the simulation, and anindication of whether each controller 140 received the expectedresponses from other controllers 140 during the simulation.

In some implementations, the result data analyzer 117 can correlate theinformation received from the SIEM device 160 with the informationreceived from the controllers 140. For example, the result data analyzer117 can determine that the metadata received from one of the controllers140 indicates that it did not receive a response that it expected duringa simulation. The result data analyzer 117 can then examine the datareceived from the SIEM device 160 to determine why the expected responsewas not received. For example, the result data analyzer 117 maydetermine based on the data received from the SIEM device that theexpected response was not received because the packets corresponding tothe expected response were blocked by one of the firewalls 150 a and 150b or by one of the IPSs 155 a-155 d. In some implementations, the resultdata analyzer 117 also can be configured to produce graphical outputcorresponding to the result data, which may be provided to anadministrator. Examples of graphical user interfaces for displaying suchresult data are described further below.

FIG. 1C shows a block diagram of an example controller 140 included inthe system network of FIG. 1A. The controller 140 can include asimulation data transceiver 142, a simulation data verifier 144, ametadata generator 146, and a database 148. As discussed above, thecontroller 140 can execute simulated attacks and other maliciousbehaviors directly within the enterprise network 105. The enterprisenetwork 105 may include several instances of the controller 140 (e.g.,at least one instance of the controller 140 in each zone of theenterprise network 105), and all of the controllers 140 may have astructure similar to that shown in FIG. 1C.

In general, the controller 140 can be a self-contained device orsoftware application that remains separate from the production equipmentused in the enterprise network 105. This configuration can help toimprove the security of the enterprise network 105, because thecontrollers 140, which execute simulated attacks and therefore may beassociated with malicious network traffic, do not have to be relied uponto also serve as production equipment. Thus, no production equipmentwithin the enterprise network 105 is put at risk. In someimplementations, the controller 140 can be implemented as a virtualmachine (VM). For example, the controller 140 can be a security hardenedVM that executes on a computing device within the enterprise network105. In other implementations, the controller 140 can be a physicalcomputing device. Whether implemented as a VM or a physical computingdevice, the controller 140 can include or can execute an operatingsystem selected to emulate a particular type of computing device forsimulations. For example, the controller 140 may execute a Windowsoperating system, a Mac OS X operating system or any other operatingsystem corresponding to a device that is to be emulated.

In some implementations, the operating system of the controller 140 canbe independent of the type of computing device and/or attack simulationor behavior to be emulated. For example, the controller 140 may run aLinux operating system, but still may emulate a Windows computing deviceduring a simulation by sending data packets that are formatted as ifthey originated from a device executing a Windows operating system. Soregardless of the type of operating system being deployed or used forthe VM, the controller may be designed, constructed, or implemented tosimulate an attack and/or behavior of an operating system other than ordifferent from the operating system of the VM. For example, the VM maycomprise a Linux operating system, but the controller may be implementedto carry out an attack simulation for a MAC OS attack.

The controller 140 can maintain a relatively aggressive securityprofile. For example, in some implementations, the controller 140 can beconfigured to refuse connections from all other computing devices at alltimes except when the controller 140 is executing a simulation. In someother implementations, the controller 140 can be configured tocommunicate only with a limited number of other computing devices, suchas the planner 115. Limiting the number of computing devices in this waycan help to prevent an outside attacker from accessing the controller140 without authorization. During the execution of a simulation, thecontroller 140 can be configured to communicate only with othercontrollers 140 within the enterprise network 105, such that thecontroller 140 is unable to communicate with production computingdevices in the enterprise network. Such a configuration can prevent theproduction computing devices from accidentally receiving maliciousnetwork traffic that may be transmitted between controllers as part of asimulation.

The simulation data transceiver 142 can be configured to transmit andreceive data related to the execution of simulations. For example, thesimulation data transceiver 142 can receive simulation management datafrom the planner 115. As discussed above, simulation management data caninclude instructions for executing various simulations, such asinformation relating to the network traffic that should be transmittedby the controller 140 during a simulation, the destination to which thenetwork traffic should be sent, and the types of responses that shouldbe expected. In some implementations, the simulation data transceiver142 can be configured to “pull” such management data from the planner115, and to drop packets that are “pushed” to the simulation datatransceiver 142. That is, the simulation data transceiver 142 may beconfigured to request such management data from the planner 115, and torefuse any data that purports to come from the planner 115 unless thedata is sent in response to a request. This configuration can help toprevent attacks from devices that may attempt to attack the controller140 by impersonating the planner 115. In some other implementations, thesimulation data transceiver 142 can be configured to receive managementdata that is pushed from the planner 115.

The simulation data transceiver 142 also can be configured to send andreceive network traffic to and from other controllers 140 in theenterprise network 105. For example, during a simulation, the simulationdata transceiver 142 can connect to other controllers 140 to simulate anattack. In some implementations, the simulation data transceiver 142 canopen a connection to another controller 140 via routing and switchingdevices such as the router 145. In some implementations, the simulationdata transceiver 142 can maintain a stateful connection with suchdevices, in the same manner as would be done by another computing devicein the same network zone. The simulation data transceiver 142 can sendsimulation data using any type of protocol, including TCP or otherprotocols that require a stateful connection. In some implementations,information corresponding to the data to be sent during a simulation canbe stored in the database 148, and the simulation data transceiver 142can retrieve that data from the database 148. The simulation datatransceiver 142 also can be configured to receive network traffic fromanother controller 140 during a simulation. For example, the simulationdata transceiver 142 can receive responses from another controller 140after sending a request to the other controller 140.

As discussed above, simulation data can be extracted or generated basedon a PCAP file representing a conversation between two host devices. Insome implementations, the simulation data transceiver 142 can beconfigured to send simulation data to a second controller 140 based onthe applications involved in the original conversation. For example, theoperating system of the controller 140 can include an implementation ofa TCP/IP stack, and the simulation data transceiver 142 can make use ofthis stack when executing a simulation whose original conversation wasconducted using an application that requires TCP communication. Thesimulation data transceiver 142 also may have additional functionalityto increase the realism of simulation data and to assist withtransmitting simulation data to a second controller 140 through thecomplex topology of the enterprise network. For example, the simulationdata transceiver 142 may support network address translation (NAT) andproxies with and without authentication, such as Basic, Radius, NTLM,and Kerberos. Thus, before a message is sent by the simulation datatransceiver 142, the simulation data transceiver 142 can be configuredto adjust the message as needed based on the topology of the enterprisenetwork. For example, the simulation data transceiver 142 may addKerberos service ticket information for proxy authentication into anHTTP packet header, if such information is required by the enterprisenetwork.

In some implementations, the simulation data transceiver 142 can beconfigured to open an encrypted channel between the controller 140 andanother controller 140 when running a simulated attack. An encryptedchannel for attack simulations can better simulate a real attack withinthe enterprise network 105, because some attacks rely on using encryptednetwork traffic to avoid being detected or blocked by securitymechanisms. Thus, by sending encrypted network traffic during an attack,the security mechanisms in the enterprise network can be more thoroughlytested. In some implementations, if the security mechanisms, such as theIPSs and firewalls, are not decrypting and inspecting encrypted networktraffic, an attack may be successfully carried out in the enterprisenetwork. For example, certain security mechanisms should act as a securesocket layer (SSL) man-in-the-middle proxy to decrypt network trafficand direct the decrypted network traffic through an IPS or IDS. By usingencrypted traffic for some attack simulations, this functionality can betested and an administrator can be notified if the encrypted attacksimulation goes undetected, so that the problem can be corrected. Insome implementations, the simulation data transceiver 142 also can beconfigured to establish a second channel between the controller 140 andthe other controller 140 that is participating in a simulation. Thesecond channel can be used to send management data, which may includecommands and configuration information that is separate from the actualsimulated attack data. Separating the management data from the attackdata can provide a more realistic attack simulation, because theencrypted first channel is used only for attack data, as would occurduring an actual attack.

The simulation data verifier 144 can be configured to determine whetherthe network traffic received from another controller 140 during asimulation corresponds to the expected responses. In someimplementations, the database 148 can store information corresponding tothe responses that are expected to be received during a simulation. Thesimulation data verifier 144 can retrieve this information from thedatabase 148 and, as network traffic is received from another controller140 during a simulation, the simulation data verifier 144 can comparethe received data to the expected data. As discussed above, theapplication associated with a particular simulation can impact theexpected responses for the simulation, as the higher level applicationdata is more important than the lower level packet data. For example,for an HTTP simulation, the simulation data verifier 144 can beconfigured to ignore the information contained in packet headers andinstead may only examiner the body of response packets, because thepacket headers may be altered in various ways by intermediary devices inthe enterprise network. The metadata generator 146 can receive theresults of the comparisons performed by the simulation data verifier144, and can produce metadata based on the results. For example, in someimplementations, the metadata generator 146 can generate a flagindicating a mismatch for each instance of network traffic received fromanother controller 140 that does not match the data that was expected tobe received. The simulation data transceiver 142 can send the metadataproduced by the metadata generator 146 to the planner 115 after thesimulation has been completed, so that the planner 115 can use themetadata to analyze the results of the simulation as discussed above.

FIG. 2 is a flowchart of an example method 200 for evaluating securityin a computer network. In brief overview, the method 200 includesreceiving, at a first controller, simulation data from a planner (step210), sending, by the first controller, first network traffic to asecond controller based on the simulation data received from the planer(step 220), receiving, by the first controller, second network trafficfrom the second controller (step 230), comparing the received secondnetwork traffic to expected network traffic based on the simulation data(step 240), and sending, by the first controller, third network trafficto the planner based on the comparison of the received second networktraffic to the expected network traffic (step 250).

Referring again to FIG. 2, and in greater detail, the method 200includes receiving, at a first controller, simulation data from aplanner (step 210). In some implementations, the first controller can bepositioned within a selected zone of an enterprise network, such as theenterprise network 105 shown in FIG. 1A, or outside of the enterprisenetwork, such as in the external cloud network 110 shown in FIG. 1A. Theplanner also can be located either within the enterprise network, or canbe outside of the enterprise network in a remote cloud network. Thesimulation data can include information such as the sequence of networkpackets that should be sent from the first controller during thesimulation, as well as metadata that can facilitate execution of thesimulation, such as data indicating the types of responses that shouldbe received during the simulation and data indicating the destination towhich network traffic should be sent during the simulation. In someimplementations, the simulation data also can include management dataassociated with a second controller to be involved in the simulation. Asdiscussed above, in some implementations, all of the management data fora simulation can be sent to a single controller, and that controller candisseminate the appropriate portions of the management data to thesecond controller involved in the simulation. In some implementations,the first controller can include an interface dedicated to receivingsuch simulation data from the planner.

The method 200 also includes sending, by the first controller, firstnetwork traffic to a second controller based on the simulation datareceived from the planer (step 220). As discussed above, the simulationdata can include metadata including an indication of a destination forthe first network traffic, as well as actual data to be sent. The firstcontroller can parse the simulation data to determine the first networkdata and the intended destination. In some implementations, the firstcontroller can create a stateful, routable socket between itself and thesecond controller. For example, the first controller can includesoftware configured to handle various types of proxies (with and withoutauthentication) firewalls, network address translation (NAT), or anyother type of network configuration that devices in the enterprisenetwork must interact with in order to send and receive network traffic.Thus, the first controller can interact with the second controller in amanner similar to that which would be used by any other form ofproduction equipment, such as workstations and servers, within theenterprise network. This arrangement can increase the realism of asimulation. After the first controller establishes a connection with thesecond controller, the first controller can begin sending the firstnetwork traffic to the second controller. As discussed above the firstnetwork traffic can be indistinguishable from actual malicious data thatwould be sent during an attack. In some implementations, the firstnetwork traffic also may include simulation management data thatprovides instructions to the second controller regarding how to executethe attack simulation.

The method 200 also includes receiving, by the first controller, secondnetwork traffic from the second controller (step 230). In someimplementations, the second network traffic can be sent responsive tothe first network traffic. For example, the first network traffic caninclude a request sent from the first controller to the secondcontroller, and the second controller can be configured to send thesecond network traffic representing a response to the request.

The method 200 also includes comparing, by the first controller, thereceived second network traffic to expected network traffic based on thesimulation data (step 240). As discussed above, the first controller maystore metadata indicating the responses that it expects to receive fromthe second controller. After the second network traffic is received fromthe second controller, the first controller can compare the receivedsecond network traffic to expected network traffic to determine a matchor mismatch. A match may be determined if, for example, a source addressof the second network traffic matches the expected source address andthe payload of each packet in the second network traffic matches theexpected packet payloads. Otherwise, a mismatch may be determined if thesecond network traffic did not originate at the expected destination ordid not include the expected packet payloads. For example, a mismatchmay occur when a security mechanism positioned between the first andsecond controllers, such as a firewall or IPS, intervenes to droppackets associated with the second network traffic in a response to adetermination that the second network traffic may be malicious.

The method 200 also includes sending, by the first controller, thirdnetwork traffic to the planner based on the comparison of the receivedsecond network traffic to the expected network traffic (step 250). Insome implementations, the first controller may generate metadata basedon the comparison performed in step 240. For example, if a mismatch isdetermined, the first controller may generate metadata including anindication of why the mismatch occurred (e.g., whether the mismatch wasa result of an incorrect source address or an incorrect packet payload),as well as additional data such as the time at which the mismatchoccurred. All of this information can be sent to the planner by thefirst controller.

In some implementations, the planner also can receive similar metadatafrom the second controller and from a SIEM device that aggregates datafrom the various security mechanisms in the enterprise network. Based onthis data received by the planner, the planner can generate simulationresults that provide an indication of how the network responded to thesimulated attack. Because the planner receives data from many differentdevices in the enterprise network, the planner can analyze the receiveddata to determine holistically how the enterprise network handled thesimulation. For example, the planner can determine not only how theendpoints (i.e., the first and second controllers) responded during thesimulation, but also whether and how any intermediary devices, such asfirewalls and IPSs, intervened to prevent or detect the simulatedattack. The planner can then aggregate this data to provide anadministrator with a complete description of how the enterprise networkresponded to the attack. In some implementations, the planner mayprovide the simulation results to the administrator using one or moregraphical user interfaces. Examples of such graphical user interfacesare described further below.

In some implementations, the process 200 can be repeated multiple timesin the enterprise network to simulate different types of attacks. Forexample, the process 200 can be repeated using a different set ofcontrollers as the first and second controllers, in order to simulate anattack between a different pair of endpoints. In this way, anadministrator can run configurable attack simulations across variouspaths through the network, to gain a holistic understanding of thesecurity posture of the network. In some implementations, the process200 may be repeated on a periodic basis, so that simulations are runover time to ensure that the security posture of the enterprise networkis understood over time as network configurations may change. In someother implementations, the process 200 may be repeated when triggered byan event, such as a reconfiguration of a security mechanism. Forexample, an administrator may run a simulation that identifies asecurity flaw, and may subsequently update the configuration of one ormore security mechanisms in the enterprise network to attempt to fix theflaw. In order to confirm whether the configuration change corrected theflaw, the simulation can be run again after the reconfiguration of theone or more security mechanisms, and the result of the second simulationcan help the administrator to understand whether the reconfigurationactually corrected the flaw identified in the first simulation.

B. Graphical User Interfaces (GUIs)

FIG. 3 is an example graphical user interface (GUI) 300 for displaying anetwork topology. In some implementations, the GUI 300 can be generatedby a central device such as the planner 115 shown in FIG. 1A. The GUI300 includes a top menu bar 305, a side menu bar 310, and a networkdiagram 315. As shown, the “topology” tab of the top menu bar 305, andthe “map” tab of the side menu option, can be selected by a user tocause the network diagram 315 to be displayed. In some implementations,the network diagram 315 can be a visual representation of an enterprisenetwork whose security posture is tested by the planner and a pluralityof controllers. The network diagram 315 can show the various networkzones that exist with the enterprise network, such as the DMZ zone, theHQ users zone, the Remote Users zone, and the Internal Servers zone. Inaddition, the network diagram 315 can include various nodes within eachzone, and each node can represent a controller configured to executesimulations of attacks and other malicious behavior as discussed above.

Links in the network diagram 315 can be used to show all of theavailable communication paths within the enterprise network, so thatsimulations can be executed along paths that actually exist within theenterprise network. For example, the network diagram 315 shows that adirect network path exists between the HQ Users zone and the InternalServers zone, but that there is no direct path between the HQ Users zoneand the Remote Users zone. In some implementations, the planner can usethis information to determine the paths to be used for simulations. Forexample, if the planner intends to execute a simulated attack betweenthe HQ Users zone and the Remote Users zone, an intermediary point forsimulation data will have to be selected to accurately simulate the flowof traffic in the enterprise network, because there is no direct pathbetween the HQ Users zone and the Remote Users zone.

FIG. 4 is an example GUI 400 for selecting a particular maliciousbehavior to be simulated in a computer network. The GUI 400 includes thetop menu bar 305 (similar to that shown in FIG. 3), a dimension filtersmenu 405, an actions menu 410, and an action preview pane 415. As shown,the “simulations” tab of the top menu bar can be selected in order tocause the other menus shown in FIG. 4 to be displayed. The dimensionfilters menu 405 includes a behavior type dropdown menu, which displaysa number of a different malicious behavior types that can be simulatedto test how a network responds to each type of malicious behavior. Eachof the behavior types may include sub-behavior types. For example, theweb attack behavior type includes sub-behavior types for general,command injection, CSRF, SQL injection, and XSS. A user may select anynumber of the behavior types from the dimension filters menu 405, andinformation corresponding to each selected behavior type is then displayin the actions menu 410. For example, the actions menu 410 can displayparticular types of attacks that correspond to the selected behaviortypes. Selecting a particular action from the actions menu can causeadditional details about that action to be selected in the actionprevious menu 415. To run a simulation based on the selected action, auser can select the run button 420.

FIG. 5 is an example GUI 500 for displaying an attack sequence and anassociated path through a computer network. After a sequence of one ormore attacks is selected using the GUI 400 shown in FIG. 4, the GUI 500can be generated to display additional information about the attacksequence. The GUI 500 includes the top menu bar 305, a sequence menu bar505, and a network diagram 510. The sequence menu 505 shows an orderedlist of the types of attacks to be simulated in the enterprise network.From the sequence menu 505, a user may view details regarding thesequence of attacks to be run, and may remove and reassign thecontrollers associate with the sequence by selecting the “clear” buttonassociated with the particular attacks. The network diagram 510highlights the network path across which the simulations associated withthe attack sequence will be run. For example, as shown in FIG. 5, theattack sequence displayed in the sequence menu 505 is to be run along apath from a node in the Internal Servers zone (labeled MonsterNode1),through the Internet to a remote node labeled AWSTestNode.

In some implementations, the GUI 500 can be used to perform aconfigurable sequence of attack simulations in the network. For example,multiple attack groups can be selected to be simulated across multiplenetwork paths, and the timing of each attack sequence may be specifiedor may be conditioned as dependent upon the outcome of previous attacks.In one example, the groups of attacks shown in the sequence menu 505 canbe selected to be performed along the highlighted path of the network510, and additional groups of attacks may be selected to be performedalong the same or different network paths. In some implementations,after the first attack sequence, in which the AWSTestNode is theintended target, has completed, a second attack sequence can beperformed in which the AWSTestNode is the attacker. Such an attacksequence can be used to simulate a scenario in which an attacker takescontrol of the AWSTestNode computing device, and subsequently uses theAWSTestNode computing device to initiate future attacks against othernodes in the network.

In the example discussed above, the second attack sequence may beselected or configured to target a node having an operating systemdifferent from the operating system associated with either or both ofthe MonsterNode1 and the AWSTestNode. For example, the MonsterNode1 maybe associated with a Linux operating system, and the AWSTestNode may beassociated with a MAC OS X operating system. Thus, the first attackexecuted between these nodes may take this information into account, forexample by attempting to exploit a flaw known to exist in the operatingsystem of the target node (i.e., the AWSTestNode). A subsequentsimulated attack sequence made from the AWSTestNode may be selected orconfigured to target a node having a different operating system, such asWindows, and the subsequent attack sequence may therefore differ fromthe first attack sequence in that the subsequent attack sequence isconfigured to exploit a flaw known to exist in the Windows operatingsystem. In some implementations, all of the information for multipleattack sequences can be downloaded by the controllers in the networkfrom the planner before the attack simulations are begun. Thus, thecontrollers can already have the information required to execute acomplex series of attack sequences one after another, without requiringadditional time to download the necessary simulation data from theplanner.

In some implementations, an administrator may specify a time delaybetween the completion of a first attack sequence and the beginning of asecond attack sequence, or between the completion of any individualevents within a given attack sequence. Thus, in the example discussedabove, the ASWTestNode may wait a predetermined amount of time after thefirst attack sequence before initiating a second attack sequence againstanother node in the network 510. In some other implementations, theadministrator may condition the execution of the second attack sequenceon a particular result of the first attack sequence. For example, theadministrator may specify that the second attack sequence is only to beperformed if the first attack sequence is not detected or blocked bysecurity mechanisms in the network 510. In still other implementations,the administrator may configure a sequence of attacks to be performedaccording to various policies. For example, such policies may take intoaccount factors such as the duration of an earlier attack, the outcomeof an earlier attack, the zones in which the attacker and target for anearlier attack were located, or the time of day at which an earlierattack was successfully executed.

In some implementations, the GUI 500 can be used to ensure that only asubset of node pairings can be selected for a given attack. For example,the GUI 500 can show the network links that actually exist in theenterprise network, and can prevent an attack simulation from beingperformed between two nodes that are not joined by a network link. Thus,in the example network 510 shown in FIG. 5, the MonsterNode1 cannotexecute an attack simulation directly with the AWSTestNode1 without alsogoing through the Internet network, because no such path exists in thenetwork 510.

FIG. 6 is a first example GUI 600 for displaying results of an attacksequence simulation in a computer network. The GUI 600 can be displayed,for example, after the attacks shown in the GUI 500 of FIG. 5 have beenrun. The GUI 600 includes a job info menu 605, a sim actions menu 610, astage of attack menu 615, and a group menu 620. The job info menu 605provides an overview of the results for the attack sequence, includingits status as complete or incomplete, a name for the simulationsequence, a time at which the simulation was submitted, and anidentification of the administrator who submitted the attack sequence.The sim actions menu 610 shows a graphical representation (e.g., a piechart) or the number of simulated attacks that passed (i.e., weredetected or blocked) and the number of simulated attacks that failed(i.e., were executed in the enterprise network but were not detected orblocked). The stage of attack menu 615 shows the various stages of eachattack, including stages corresponding to recon, deliver, exploit,execute, control, and action on target. For each stage, the stage ofattack menu displays a color that may indicate whether the enterprisenetwork properly defended against that particular stage of attack. Forexample, green may indicate that the attack was successfully blocked ata particular stage. The group menu 620 can display a list of each groupof attacks that was run during a simulation, as well as in indication ofwhether each group of attacks was successfully blocked and whether eachgroup of attacks generated a security event (e.g., a detection ofmalicious traffic by a security mechanism in the enterprise network).

FIG. 7 is a second example GUI 700 for displaying results of an attacksequence simulation in a computer network. The GUI 700 includes anactive security monitors indication 234, a unique checks indication 710,and an active security alerts indication 715. In addition, the GUI 700includes a menu 720 for displaying the monitors by attack type, a menu725 for display the monitors by attack stage, and a menu 730 fordisplaying additional information for those simulations that passed. Themenu 720 provides a pie chart showing the relative proportion ofsecurity monitors in the network for handling various types of attacks,including web-type attacks, scan/enum attacks, malicious file attacks,general type attacks, and auth2 attacks. The menu 725 includes anindication of the number of monitors that are configured to detect orblock various stages of an attack, including recon, weaponize, deliver,exploit, control, execute, and persist stages. Each stage includes anumeric value representing the number of active monitors for that stage.The menu 730 displays a pie chart showing the relative proportion ofways in which simulations generated a passing result in the enterprisenetwork. For example, simulations may generate a passing result byrefusing a connection associated with a simulated attack, by generatingan event such as a flag indicating that an attack likely took place, orby generating an event and also blocking the attack that corresponds tothe event.

FIG. 8 is a third example GUI 800 for displaying results of an attacksequence simulation in a computer network. The GUI 800 includes adimensional analysis heat map 805, a columns menu 810, a rows menu 815,a cell rank indication 820, a filter menu 825, and a passing percentageindication 830. The columns menu 810 and the rows menu 815 can includean indication of the labels for the columns and rows of the heat map805, respectively. In the example shown in FIG. 8, the columns of theheat map 810 indicate the target and attacker nodes for a simulation,and rows indicate the type of attack for the simulation. The heat map805 can include a cell for each row/column pair. Thus, each cell in theheat map corresponds to a particular attack type for a particular set oftarget and attack nodes. In some implementations, the color of each cellcan correspond to the relative performance of that cell. For example, insome implementations, a lighter color may indicate that the cellrepresents a type of attack and set of node pairs for which a relativelylarge percentage of simulations passed, while a darker color mayindicate that the cell represents a type of attack and set of node pairsfor which a relatively small percentage of simulations passed. Cells canbe selected by a user, and the relative rank of the selected cell can bedisplayed in the cell rank indication 820. The user also can customizethe heat map 805 by selecting various filters in the cell filter menu825. The passing percentage 830 displays a pie chart showing therelative percentage of simulations that passed as compared to those thatfailed.

FIG. 9 is a fourth example GUI 900 for displaying results of an attacksequence simulation in a computer network. The GUI 900 includes a seriesof reporting options 905, a summary 910, and a total actions run menu915. The reporting options menu 905 can include a plurality of dropdownoptions for customizing the results that are displayed by the GUI 900.For example, the reporting options menu 905 can allow a user to select atime frame as well as the types of results to be displayed for that timeframe. The summary 910 can include information relating to the number ofsimulations that have been run and the detection rate for various typesof simulations based on the method they used to transfer data. The totalactions run menu 915 can display a pie chart showing the relativeproportions of simulations run for each transfer method.

C. Controller-to-Controller Communication

As described above, a system such as the system 100 shown in FIG. 1 canbe configured to facilitate comprehensive evaluation of the securityposture of the production network. In some implementations, testing ofthe security posture of the production network can be carried out by aplanner and one or more controllers. Because the controllers arepositioned within the enterprise network, the tests performed by thecontrollers can accurately reflect the complexity of the enterprisenetwork better than isolated laboratory testing of individual componentstaken out of the network topology. Furthermore, because tests do notrely on communications between the production computing devices in theproduction network, the production computing devices are never put atrisk during testing. In some implementations, the controllers serve onlyto test the security of the enterprise network, and may not be used fortypical day-to-day operations of the production network.

FIG. 10 is a flowchart of an example method 1000 for controllingexecution of malicious behavior in a production network to test asecurity system of the production network. In brief overview, the method1000 includes receiving, by a first controller on a first node in aproduction network with a security system, instructions to operate as anattacker and data for executing a predetermined malicious behavior onthe production network (step 1010). A second controller on a second nodereceives, from the planner, instructions to operate as a target of thepredetermined malicious behavior by the attacker (step 1020). The method1000 includes establishing a connection between the first controller andthe second controller (step 1030). The method 1000 includestransmitting, responsive to the instructions by the first controller viathe connection to the second controller via at least a portion of thesecurity system of the production network, network traffic comprisingthe predetermined malicious behavior and generated using the data (step1040), receiving, by the first controller via the connection from thesecond controller, one or more responses to the network traffic (step1050), and determining, by the first controller, whether the one or moreresponses from the second controller are as expected (step 1060).

Referring again to FIG. 10, and in greater detail, the method 1000includes receiving, by a first controller on a first node in aproduction network with a security system, instructions to operate as anattacker and data for executing a predetermined malicious behavior onthe production network (step 1010). In some implementations, theproduction network can be a network similar to the enterprise network105 shown in FIG. 1A, and the first controller can be any of thecontrollers 140 a-140 d located within the network 105. The firstcontroller can receive the instructions from a planner, such as theplanner 115 shown in FIG. 1A. In some implementations, the planner canbe positioned outside of the production network as illustrated in FIG.1A. However, in other implementations, the planner also can be includedwithin the production network. The instructions can include any type andform of data relating to the malicious behavior to be carried out by thecontroller, which may correspond to various types of cyber attacks. Forexample, the instructions can include one or more requests for the firstcontroller to execute the malicious behavior, including information suchas a sequence of network packets that should be sent from the firstcontroller during the performance of the malicious behavior. Theinstructions also can include metadata that can facilitate execution ofthe malicious behavior. Such metadata can be any form data indicatingthe types and/or content of one or more responses that the firstcontroller should expect to receive, as well as data indicating thedestination to which network traffic should be sent during the maliciousbehavior (i.e., a network address associated with a second controller).In some implementations, the instructions also can include managementdata associated with a second controller to be involved in the executionof the malicious behavior.

At step 1020 of the method 1000, a second controller on a second nodereceives, from the planner, instructions to operate as a target of thepredetermined malicious behavior by the attacker (i.e., the firstcontroller on the first node). As described above, the second controllercan be any of the controllers 140 a-140 d in the enterprise network 105shown in FIG. 1A. In some implementations, the second controller can bea controller located within a different zone than the first controller.For example, such an arrangement can allow for the testing of amalicious behavior across different zones of the network requiring thetransmission of data packets through various networking and securitydevices, such as the IPSs 155, the firewalls 150, and the router 145shown in FIG. 1A. The second controller can receive instructions thatare similar (or complimentary) to the instructions received by the firstcontroller in step 1010. For example, the instructions can includeinformation such as a sequence of network packets that are to bereceived from the first controller during the performance of themalicious behavior by the first controller, as well as one or moreresponses that the second controller should transmit to the firstcontroller in response.

The method 1000 includes establishing a connection between the firstcontroller and the second controller (step 1030). In someimplementations, the connection between the first controller and thesecond controller can be established based at least in part on the typeof malicious behavior that the first controller has been instructed toperform. For example, the malicious behavior may require a particulartype of connection. In some implementations, the malicious behavior mayrequire a secure connection between the first controller and the secondcontroller. Thus, the established connection can be a secure connectionsuch as a secure shell (SSH) connection or a secure hypertext transferprotocol (HTTPS) connection. In some implementations, the connection canbe a stateful connection. For example, the first controller can beconfigured to handle various types of proxies, firewalls, networkaddress translation (NAT), or any other type of network configurationthat a device in the production network must interact with in order tosend and receive network traffic. Thus, the first controller caninteract with the second controller in a manner similar to that whichwould be used by any other form of production equipment, such asworkstations and servers, within the production network. In someimplementations, the connection can be an encrypted connection. In someother implementations, any other type or form of connection can beestablished. Regardless of the type of connection established, theconnection can allow an attack to be replicated between the firstcontroller and the second controller, as described below.

The method 1000 includes transmitting, responsive to the instructions bythe first controller via the connection to the second controller via atleast a portion of the security system of the production network,network traffic comprising the predetermined malicious behavior andgenerated using the data (step 1040). The instructions that the firstcontroller received at step 1010 can include information including anindication of a destination for the first network traffic (i.e., thesecond controller), as well as the actual data to be sent. The firstcontroller can parse this information to determine the network trafficand the intended destination, and can transmit the network traffic, forexample via a network interface included within the first controller. Insome implementations, the network traffic can be indistinguishable fromactual malicious data that would be sent during an attack. For example,the network traffic can include a request for the second controller toperform the malicious behavior under test.

The method 1000 includes receiving, by the first controller via theconnection from the second controller, one or more responses to thenetwork traffic (step 1050). In some implementations, the secondcontroller can send the one or more responses based in part on thenetwork traffic it received from the first controller. For example, thenetwork traffic sent by the first controller can include a request, andthe second controller can be configured to send the one or moreresponses in response to the request. In some implementations, thesecond controller can generate the one or more responses based in parton the instructions it received (e.g., from the planner) in step 1020.

The method 1000 also includes determining, by the first controller,whether the one or more responses from the second controller are asexpected (step 1060). In some implementations, the first controller canmake this determination based on a comparison of the one or moreresponses with data received from the planner in step 1010. For example,as discussed above, the instructions received at the first controller instep 1010 can include expected response data corresponding to theresponses that the first controller expects to receive from the secondcontroller. After the one or more responses are received from the secondcontroller in step 1050, the first controller can compare the one ormore responses to the expected data to determine whether the one or moreresponses are as expected. The first controller can determine that theone or more responses are as expected if, for example, a source addressof the responses matches an expected source address and if the payloadof each packet in the one or more responses matches expected packetpayloads. On the other hand, the first controller can determine that theone or more responses are not as expected if the one or more responsesdid not originate at the second controller or if they do not include theexpected packet payloads. For example, a security mechanism positionedbetween the first and second controllers, such as a firewall or IPS,could intervene to drop packets associated with the second networktraffic in a response to a determination that the second network trafficmay be malicious, thereby leading to an unexpected response (or noresponse) at the first controller.

In some implementations, if the first controller determines a mismatchbetween the one or more responses and the expected response data, thefirst controller can be configured to generate metadata about themismatch, and can communicate this metadata to the planner. In someimplementations, both the first controller and the second controller canbe configured to maintain the network traffic representing the maliciousbehavior between the first controller and the second controller, forexample by storing some or all of the network traffic in respectivememory elements on each of the first controller and the secondcontroller. In some implementations, the method 1000 can be repeatedmultiple times. Repeating the method 1000 can allow the security postureof the production network to be continuously monitored, as configurationchanges on any of the devices within the network may alter the securityposture of the network over time. Thus, in some implementations, anadministrator may configure the planner to perform the method 1000 withdifferent pairs of controllers on a periodic basis.

D. Attack Simulation Sequences

Sophisticated malicious behavior in a computer network often relies onone or more sequences of multiple attacks. These attack sequences mayoccur between different node pairs and across different paths throughthe network. In addition, the results of an attack may impact theexecution of a subsequent attack in the sequence. Thus, it can bedifficult to test this type of complex malicious behavior in an isolatedlaboratory environment, as the conditional behavior that may depend onthe results of attack sequences occurring across different node pairsand different network paths may not be readily replicated. To more fullytest the security posture of a production network, attack simulationsequences can be run directly within the production network.

FIG. 11A is a flowchart of an example method for configuring acontrolled execution of a sequence of attacks in a production network totest a security system. In brief overview, the method 1100 includesreceiving, via a planner, a first specification of a first attack of aplurality of attacks in an attack sequence (step 1105), receiving, viathe planner, a second specification of a second attack of the pluralityof attack in the attack sequence (step 1110), receiving, by the firstcontroller, second network traffic from the second controller (step1115), storing, by the planner, the attack sequence including the firstspecification of the first attack and the second specification of thesecond attack (step 1120), and identifying, by the planner, the storedattack sequence for controlled execution within the production networkto test at least a portion of a security system (step 1125).

Referring again to FIG. 11A, and in greater detail, the method 1100includes receiving, via a planner (e.g., the planner 115 in FIG. 1A), afirst specification of a first attack (e.g., Group 1 in FIG. 5) of aplurality of attacks (e.g., Groups 1-4 in FIG. 5) in an attack sequence(e.g., an attack sequence of Groups 1-4 in the menu bar 505 in FIG. 5)(step 1105). In some implementations, referring to FIG. 5, the firstspecification (e.g., Group 1) can identify a first attack node (e.g.,Node111 c) and a first target node (e.g., MonsterNode1) in a productionnetwork, a first network path selected between the first attack node andthe first target node, and a first predetermined malicious behavior(e.g., rapid port scan & OS detection). In some implementations, each ofthe first and second attack nodes may be a controller (e.g., thecontrollers 140 in FIG. 1A) deployed within a zone of the productionnetwork. In some implementations, the first specification of the firstattack can be received by receiving a selection of the first networkpath via interaction with a graphical representation of links betweennodes (e.g., the network diagram 510 in FIG. 5) in the productionnetwork.

The method 1100 also includes receiving, via the planner, a secondspecification of a second attack (e.g., Group 4 in FIG. 5) of theplurality of attack in the attack sequence (e.g., the attack sequence ofGroups 1-4 in the menu bar 505 in FIG. 5) (step 1110). In someimplementations, referring to FIG. 5, the second specification (e.g.,Group 4) can identify a second attack node (e.g., MonsterNode1) and asecond target node (e.g., AWSTestNode) in the production network, asecond network path (e.g., MonsterNode1 to Internal Servers to Internetto AWSTestNode as shown in the network diagram 510) selected between thesecond attack node and the second target node, and a secondpredetermined malicious behavior (e.g., data exfiltration via FTP). Insome implementations, referring to FIG. 5, the second specification(e.g., Group 4) can identify the second predetermined malicious behavior(e.g., data exfiltration via FTP) different than the first predeterminedmalicious behavior (e.g., rapid port scan & OS detection). In someimplementations, the second specification can identify the second attacknode and the second target node including types of operating systems(e.g., a MAC OS X operating system) different than at least one of thefirst attack node or the first target node (e.g., the Windows operatingsystem).

The method 1100 also includes receiving, via the planner, identificationof one or more conditions upon which to execute the second attacksubsequent to execution of the first attack in the attack sequence (step1115). For example, referring to FIG. 5, after the first attack (orattack sequence), in which the AWSTestNode is the intended target, hascompleted, a second attack (or a second attack sequence) can beperformed in which the AWSTestNode is the attacker. Such an attack canbe used to simulate a scenario in which an attacker takes control of theAWSTestNode computing device, and subsequently uses the AWSTestNodecomputing device to initiate future attacks against other nodes in thenetwork.

In some implementations, the identification of the one or moreconditions can be received by identifying the one or more conditions asa time duration between the first attack and the second attack. Forexample, referring to FIG. 5, the ASWTestNode may wait a predeterminedamount of time after a first attack (or a first attack sequence) beforeinitiating a second attack (or a second sequence) against another nodein the network 510.

In some implementations, the identification of the one or moreconditions can be received by identifying the one or more conditions asdetection of one of failure of or success of the first attack. Forexample, the administrator may specify that the second attack (or attacksequence) is only to be performed if the first attack (or attacksequence) is not detected or blocked by security mechanisms in thenetwork 510 (see FIG. 5).

The method 1100 also includes storing, by the planner (e.g., the planner115), the attack sequence including the first specification of the firstattack and the second specification of the second attack (e.g., Group 1and Group 4 in FIG. 5) (step 1120). For example, the attack sequence canbe received by the planner 115 and stored in the database 118 (see FIG.1B).

The method 1100 also includes identifying, by the planner (e.g., theplanner 115), the stored attack sequence for controlled execution withinthe production network to test at least a portion of a security system(step 1125). For example, referring to FIG. 1B, the simulation datamanager 116 can retrieve an attack sequence from the database 118 andcan process the attack sequence to generate simulation instructions thatwill cause the controllers 140 to send network traffic to replicate theattack upon a portion of a security system. In some implementations, allof the information for multiple attack sequences (e.g., the attacksequence shown in FIG. 5) can be downloaded by the controllers (e.g.,the controllers 140 a-140 e in FIG. 1A) in the network from the plannerbefore the attack simulations are begun. Thus, the controllers canalready have the information required to execute a complex series ofattack sequences one after another, without requiring additional time todownload the necessary simulation data from the planner.

In some implementations, the method 1100 includes receiving a selectionof a pairing of a first attack node and a second attack node or a paringof an attack node and a target node via interaction with a graphicalrepresentation of a topology of the production network. In someimplementations, referring to FIG. 5, the GUI 500 can be used to ensurethat only a subset of node pairings such that there exists a path in thenetwork between each node pairing can be selected for a given attack.For example, in the example network 510 shown in FIG. 5, the pairing ofMonsterNode1 (as an attack node) and AWSTestNode1 (as a target node) canbe selected such that there exists a path between the two nodes in thenetwork diagram 510. That is, the GUI 500 can show the network linksthat actually exist in the enterprise network, and can prevent an attacksimulation from being performed between two nodes that are not joined bya network link. Thus, in the example network 510 shown in FIG. 5, theMonsterNode1 cannot execute an attack simulation directly with theAWSTestNode1 without also going through the Internet network, because nosuch path exists in the network 510. In some other implementations, theplanner can receive the pairing of a first attack node and a secondattack node or the pairing of an attacker node and a target node in adifferent manner that may not require any graphical representation ofthe topology of the production network. For example, the planner canreceive this information by way of an API configured to allow a user(e.g., a network administrator) to select the pairing of a first attacknode and a second attack node and/or the pairing of an attacker node anda target node, and to transmit this information to the planner, withoutthe use of any graphical representation of the topology of the network.In some implementations, the pairs of nodes can be specified, forexample, in a text-based format or other non-graphical format.

In some implementations, in a case where an attack sequence can be usedto simulate a scenario in which an attacker (as a first attack node)takes control of a first computing device (as a first target node), andsubsequently uses the same first computing device (as a second attacknode) to initiate future attacks against other nodes (as a second targetnode) in the network, the pairing of the first attacker and the secondattacker can be selected such that there exists a path between the twonodes in the network.

In some implementations, a system (e.g., the system 100 in FIG. 1A) forconfiguring a controlled execution of a sequence of attacks in aproduction network to test a security system can include a planner(e.g., the planner 115 in FIG. 1A) executable on a processor coupled tomemory. In some implementations, the planner can be configured toreceive a first specification of a first attack (e.g., Group 1 in FIG.5) of a plurality of attacks in an attack sequence (e.g., an attacksequence of Groups 1-4 in the menu bar 505 in FIG. 5). In someimplementations, the first specification (e.g., Group 1) can identify afirst attack node (e.g., Node111 c) and a first target node (e.g.,MonsterNode1) in a production network, a first network path selectedbetween the first attack node and the first target node, and a firstpredetermined malicious behavior (e.g., rapid port scan & OS detection).In some implementations, the first and second attack nodes may becontrollers (e.g., the controllers 140 in FIG. 1A) deployed within theproduction network. In some implementations, the planner can beconfigured to receive a second specification of a second attack (e.g.,Group 4 in FIG. 5) of the plurality of attack in the attack sequence(e.g., the attack sequence of Groups 1-4 in the menu bar 505 in FIG. 5).In some implementations, the second specification (e.g., Group 4 in FIG.5) can identify a second attack node (e.g., MonsterNode1) and a secondtarget node (e.g., AWSTestNode) in the production network, a secondnetwork path (e.g., MonsterNode1 to Internal Servers to Internet toAWSTestNode as shown in the network diagram 510) selected between thesecond attack node and the second target node, and a secondpredetermined malicious behavior (e.g., data exfiltration via FTP). Insome implementations, each of the first and second attack nodes may be acontroller (e.g., the controllers 140 in FIG. 1A) deployed within a zoneof the production network. In some implementations, the planner can beconfigured to receive identification of one or more conditions (e.g., ascenario in which an attacker takes control of a first computing device,and subsequently uses the first computing device to initiate futureattacks against other nodes in the network) upon which to execute thesecond attack subsequent to execution of the first attack in the attacksequence. In some implementations, the planner can be configured tostore (e.g., in the database 118 in FIG. 1B) the attack sequenceincluding the first specification of the first attack and the secondspecification of the second attack. In some implementations, the plannercan be configured to identify the stored attack sequence (e.g., byretrieving from the database 118 in FIG. 1B) for controlled executionwithin the production network to test at least a portion of a securitysystem.

In some implementations, the planner of the system can be furtherconfigured to receive a selection of a pairing of a first attack nodeand a second attack node or a pairing of an attacker node and a targetnode via interaction (e.g., MonsterNode1 and AWSTestNode1) with agraphical representation of a topology of the production network (e.g.,the network diagram 510 in FIG. 5). For example, in the example network510 shown in FIG. 5, the pairing of MonsterNode1 and AWSTestNode1 can beselected such that there exists a path between the two nodes exists inthe network diagram 510. In some other implementations, the planner canreceive the pairing of a first attack node and a second attack node orthe pairing of an attacker node and a target node in a different mannerthat may not require any graphical representation of the topology of theproduction network, as described above.

In some implementations, the planner of the system can be furtherconfigured to receive a selection of a first network path (e.g.,MonsterNode1 to Internal Servers to Internet to AWSTestNode1 in FIG. 5)via interaction with a graphical representation of links between nodesin the production network (e.g., the network diagram 510).

In some implementations, referring to FIG. 5, the planner of the systemcan be further configured to receive a second specification (e.g., Group4) identifying a second predetermined malicious behavior (e.g., dataexfiltration via FTP) different than a first predetermined maliciousbehavior (e.g., rapid port scan & OS detection in Group 1).

In some implementations, the planner of the system can be furtherconfigured to receive the second specification identifying the secondattack node and the second target node including types of operatingsystems (e.g., a MAC OS X operating system) different than types ofoperating systems of at least one of the first attack node or the firsttarget node (e.g., the Windows operating system).

In some implementations, the planner of the system can be furtherconfigured to receive identification of the one or more conditions as atime duration between the first attack and the second attack, e.g., anode may wait a predetermined amount of time after the first attack (orattack sequence) before initiating the second attack (or attacksequence) against another node in the network.

In some implementations, the planner of the system can be furtherconfigured to receive identification of the one or more conditions asdetection of one of failure of or success of the first attack, e.g., thesecond attack is only to be performed if the first attack is notdetected or blocked by security mechanisms in the network.

FIG. 11B is a flowchart of an example method for controlled execution ofa sequence of attacks in a production network to test at least a portionof a security system of the production network. In brief overview, themethod 1150 includes identifying, by a simulation data manager, anattack sequence (step 1155), communicating, by the simulation datamanager, instructions to the first attack node and the first target nodein the production network (step 1160), monitoring, by a result dataanalyzer, execution of the first attack between the first attack nodeand the first target node for the one or more conditions (step 1165),and determining, by the simulation data manager responsive to detectingthe one or more conditions via monitoring (step 1170).

Referring again to FIG. 11B, and in greater detail, the method 1150includes identifying, by a simulation data manager (e.g., a simulationdata manager 116 in FIG. 1B), an attack sequence including a firstattack and a second attack (e.g., attack groups 1-4 in FIG. 5) and oneor more conditions upon which to execute the second attack subsequent tothe first attack (e.g., execution of the second attack is dependent uponthe outcome of the first attack) (step 1155). In some implementations,referring to FIG. 5, the first attack (e.g., Group 1) can specify afirst attack node (e.g., Node111 c) and a first target node (e.g.,MonsterNode1) in a production network, a first network path selectedbetween the first attack node and the first target node, and a firstpredetermined malicious behavior (e.g., rapid port scan & OS detection).In some implementations, referring to FIG. 5, the second attack (e.g.,Group 4) can specify a second attack node (e.g., MonsterNode1) and asecond target node (e.g., AWSTestNode) in the production network, asecond network path selected between the second attack node and thesecond target node (e.g., MonsterNode1 to Internal Servers to Internetto AWSTestNode), and a second predetermined malicious behavior (e.g.,data exfiltration via FTP). In some implementations, each of the firstand second attack nodes may be a controller (e.g., the controllers 140in FIG. 1A) deployed within a zone of the production network. In someimplementations, the simulation data manager can identify the attacksequence by identifying the attack sequence including the first attacknode and the first target node on nodes with a type of operating system(e.g., a MAC OS X operating system) different than a type of operatingsystem at least one of the second attack node and second target node(e.g., the Windows operating system). In some implementations, referringFIG. 5, the simulation data manager can identify the attack sequenceincluding the first predetermined malicious behavior (e.g., rapid portscan & OS detection in Group 1) different than the second predeterminedmalicious behavior (e.g., data exfiltration via FTP in Group 4). In someimplementations, referring to FIG. 5, the simulation data manager canidentify the attack sequence including the first selected network path(e.g., Node111 c to MonsterNode1 in Group 1) different than the secondselected network path (e.g., MonsterNode1 to Internal Servers toInternet to AWSTestNode in Group 4).

The method 1150 also includes communicating, by the simulation datamanager, instructions to the first attack node (e.g., Node111 c in FIG.5) and the first target node (e.g., MonsterNode1 in FIG. 5) in theproduction network to initiate the first predetermined maliciousbehavior (e.g., rapid port scan & OS detection) via the first pathselected between the first attack node and the first target node (step1160). In some implementations, the simulation data manager 116 cancommunicate directly with each of the first attack node and the firsttarget node that are to be involved in a simulation. In suchimplementations, the simulation data manager 116 can send simulationdata directly to both of the first attack node and the first targetnode. For example, the simulation data manager 116 (see FIG. 1B) cancommunicate simulation execution instructions including any informationnecessary for the first attack node to carry out the first predeterminedmalicious behavior (e.g., rapid port scan & OS detection). In someimplementations, simulation instructions can include a collection of allof the requests that the first attack node or the first target nodeshould make during a simulation, as well as all of the responses thatthe first attack node or the first target node should receive during thesimulation. Simulation execution instructions can be based on an actualattack that has been carried out in the past and that could potentiallypose a threat to the enterprise network 105 (see FIG. 1A). In someimplementations, the simulation data manager 116 can generateinstructions for such a simulation based on PCAP files corresponding tothe first predetermined malicious behavior. In some implementations, thesimulation data manager can transmit, responsive to instructions fromthe planner (e.g., the planner 115 in FIG. 1A) by the first attack nodevia a connection to the first target node and via at least a portion ofthe security system of the production network, network traffic includingthe first predetermined malicious behavior. For example, the networktraffic including the malicious behavior can include information relatedto initial registration of each of the first attack node and the firsttarget node, configuration of each of the nodes, and simulationexecution instructions. In some implementations, the initialregistration can be any process that pairs each of the first attack nodeand the first target node with the planner 115. In some implementations,referring to FIG. 1A, a private key exchange can take place between eachof the first attack node and the first target node and the planner 115,which can ensure that no other computing devices can impersonate one ofthe first attack node and the first target node, which could compromisethe security of the enterprise network 105 in which the first attacknode and the first target node are deployed. In some implementations, asdiscussed above, the simulation execution instructions may include anyinformation necessary for the first attack node and the first targetnode to carry out a simulated attack, e.g., a collection of all of therequests that the first attack node or the first target node should makeduring a simulation, as well as all of the responses that the firstattack node or the first target node should receive during thesimulation.

The method 1150 also includes monitoring, by a result data analyzer(e.g., the result data analyzer 117 in FIG. 1B), execution of the firstattack between the first attack node and the first target node for theone or more conditions (step 1165). In some implementations, the resultdata analyzer also can receive information (e.g., metadata) from thefirst attack node, the first target node or other controllers (e.g., thecontrollers 140 in FIG. 1A) after a simulation has completed. Forexample, such metadata may include the start time and end time for thesimulation, the sockets and ports used during the simulation, and anindication of whether the first attack node or the first target nodereceived the expected responses from other controllers 140 during thesimulation. In some implementations, the result data analyzer 117 cancorrelate the information received from the SIEM device 160 with theinformation received from the first attack node or the first targetnode. For example, the result data analyzer 117 can determine that themetadata received from the first attack node or the first target nodeindicates that it did not receive a response that it expected during asimulation. The result data analyzer 117 can then examine the datareceived from the SIEM device 160 to determine why the expected responsewas not received. For example, the result data analyzer 117 maydetermine based on the data received from the SIEM device that theexpected response was not received because the packets corresponding tothe expected response were blocked by one of the firewalls 150 a and 150b or by one of the IPSs 155 a-155 d (see FIG. 1A).

In some implementations, the execution of the first attack can bemonitored by detecting via monitoring the one or more conditionsincluding at least one of failure or success of the first predeterminedmalicious attack. In some implementations, the result data analyzer 117also can be configured to produce graphical output corresponding to theresult data (e.g., failure or success of a simulated attack), which maybe provided to an administrator. For example, referring to FIG. 6, suchgraphical output corresponding to the result data can include agraphical representation (e.g., a pie chart) of the number of simulatedattacks that passed (i.e., were detected or 10 blocked) and the numberof simulated attacks that failed (i.e., were executed in the enterprisenetwork but were not detected or blocked), a color that may indicatewhether the enterprise network properly defended against a particularstage of attack in various stages of attack (e.g., recon, deliver,exploit, execute, control, and action on the target node), a list ofeach group of attacks that was run during a simulation, as well as anindication of whether each group of attacks was successfully blocked andwhether each group of attacks generated a security event (e.g., adetection of malicious traffic by a security mechanism in the enterprisenetwork). Referring to FIG. 7, such graphical output corresponding tothe result data can include a pie chart (e.g., the menu 720) showing therelative proportion of security monitors in the network for handlingvarious types of attacks, including web-type attacks, scan/enum attacks,malicious file attacks, general type attacks, and auth2 attacks, anindication (e.g., the menu 725) of the number of monitors that areconfigured to detect or block various stages of an attack, includingrecon, weaponize, deliver, exploit, control, execute, and persiststages. Each stage 5 includes a numeric value representing the number ofactive monitors for that stage, a pie chart (e.g., the menu 730) showingthe relative proportion of ways in which simulations generated a passingresult in the enterprise network. For example, simulations may generatea passing result by refusing a connection associated with a simulatedattack, by generating an event such as a flag indicating that an attacklikely took place, or by generating an event and also blocking theattack that 10 corresponds to the event.

In some implementations, the execution of the first attack can bemonitored by detecting via monitoring the one or more conditionsincluding a time duration. For example, referring to FIG. 9, thegraphical user interface (GUI) 900 displays the reporting options menu905 that can allow a user to select a time duration (e.g., one month) aswell as the types of results to be displayed for that time frame.Thereby, the GUI 900 can display the summary 910 including informationrelating to the number of simulations that have been run during theselected time duration and the detection rate for various selected typesof simulations based on the method they used to transfer data, and a piechart 915 showing the relative proportions of simulations run for eachtransfer method during the selected time duration.

The method 1150 also includes determining, by the simulation datamanager (e.g., the simulation data manager 116; see FIG. 1B) responsiveto detecting the one or more conditions via monitoring, to execute thesecond attack of the attack sequence (step 1170). For example, based onthe result data of the first attack (e.g., failure or success of asimulated attack) obtained by monitoring at step 1165, the simulationdata manager 116 can determine whether the second attack of the attacksequence is executed. In some implementations, the simulation datamanager 116 can determine that the second attack is only to be performedif the first attack is not detected or blocked by security mechanisms inthe network 510 (see FIG. 5).

In some implementations, the method 1150 includes communicating, by thesimulation data manager, instructions to the second attack node (e.g.,MonsterNode1 in Group 4 in FIG. 5) and the second target node (e.g.,AWSTestNode in Group 4 in FIG. 5) in the production network to initiatethe second predetermined malicious behavior (e.g., data exfiltration viaFTP in Group 4 in FIG. 5) via the second path selected between thesecond attack node and the second target node (e.g., MonsterNode1 toInternal Servers to Internet to AWSTestNode as shown in the networkdiagram 510 in FIG. 5).

In some implementations, a system (e.g., the system 100 in FIG. 1A) forcontrolled execution of a sequence of attacks in a production network totest at least a portion of a security system of the production network,can include a simulation data manager (e.g., a simulation data manager116 in FIG. 1B) executable on a processor coupled to memory, and aresult data analyzer (e.g., the result data analyzer 117 in FIG. 1B). Insome implementations, the simulation data manager can be configured toidentify an attack sequence including a first attack and a second attack(e.g., attack groups 1-4 in FIG. 5) and one or more conditions uponwhich to execute the second attack subsequent to the first attack (e.g.,execution of the second attack is dependent upon the outcome of thefirst attack). In some implementations, referring to FIG. 5, the firstattack (e.g., Group 1) can specify a first attack node (e.g., Node111 c)and a first target node (e.g., MonsterNode1) in a production network, afirst network path selected between the first attack node and the firsttarget node, and a first predetermined malicious behavior (e.g., rapidport scan & OS detection). In some implementations, referring to FIG. 5,the second attack can specify a second attack node (e.g., MonsterNode1)and a second target node (e.g., AWSTestNode) in the production network,a second network path selected between the second attack node and thesecond target node (e.g., MonsterNode1 to Internal Servers to Internetto AWSTestNode), and a second predetermined malicious behavior (e.g.,data exfiltration via FTP). In some implementations, the simulation datamanager can be configured to communicate instructions to the firstattack node (e.g., Node111 c in FIG. 5) and the first target node (e.g.,MonsterNode1 in FIG. 5) in the production network to initiate the firstpredetermined malicious behavior (e.g., rapid port scan & OS detection)via the first path selected between the first attack node and the firsttarget node. In some implementations, the result data analyzer (e.g.,the result data analyzer 117 in FIG. 1B) can be configured to monitorexecution of the first attack between the first attack node and thefirst target node for the one or more conditions (e.g., failure orsuccess of the first predetermined malicious attack during a particulartime duration). In some implementations, responsive to the result dataanalyzer (e.g., the simulation data manager 116; see FIG. 1B) detectingthe one or more conditions (e.g., failure or success of a simulatedattack), the simulation data manager can be configured to determine toexecute the second attack of the attack sequence. For example, thesimulation data manager 116 can determine that the second attack is onlyto be performed if the first attack is not detected or blocked bysecurity mechanisms in the network 510 (see FIG. 5).

In some implementations, the simulation data manager of the system canbe configured to communicate instructions to the second attack node(e.g., MonsterNode1 in Group 4 in FIG. 5) and the second target node(e.g., AWSTestNode in Group 4 in FIG. 5) in the production network toinitiate the second predetermined malicious behavior (e.g., dataexfiltration via FTP in Group 4 in FIG. 5) via the second path selectedbetween the second attack node and the first second node (e.g.,MonsterNode1 to Internal Servers to Internet to AWSTestNode as shown inthe network diagram 510 in FIG. 5).

In some implementations, the simulation data manager of the system canbe configured to identify the attack sequence including the first attacknode and the first target node on nodes with a type of operating system(e.g., a MAC OS X operating system) different than at least one of thesecond attack node and second target node (e.g., the Windows operatingsystem). In some implementations, referring FIG. 5, the simulation datamanager of the system can be configured to identify the attack sequenceincluding the first predetermined malicious behavior (e.g., rapid portscan & OS detection in Group 1) different than the second predeterminedmalicious behavior (e.g., data exfiltration via FTP in Group 4).

In some implementations, referring FIG. 5, the simulation data managerof the system can be configured to identify the attack sequenceincluding the first selected network path (e.g., Node111 c toMonsterNode1 in Group 1) different than the second selected network path(e.g., MonsterNode1 to Internal Servers to Internet to AWSTestNode inGroup 4).

In some implementations, the first attack node of the system can beconfigured to, responsive to instructions from a planner (e.g., theplanner 115 in FIG. 1A), transmit via a connection to the first targetnode and via at least a portion of the security system of the productionnetwork, network traffic including the first predetermined maliciousbehavior (e.g., information related to initial registration of each ofthe first attack node and the first target node, configuration of eachof the nodes, and simulation execution instructions). In someimplementations, as discussed above, the simulation executioninstructions may include any information necessary for the first attacknode and the first target node to carry out a simulated attack, e.g., acollection of all of the requests that the first attack node or thefirst target node should make during a simulation, as well as all of theresponses that the first attack node or the first target node shouldreceive during the simulation.

In some implementations, the result data analyzer of the system can beconfigured to detect the one or conditions including a time duration.For example, a user can select a time duration (e.g., one month) todisplay result data of attacks that have been run during the selectedtime duration (see FIG. 9).

In some implementations, the result data analyzer of the system can beconfigured to detect the one or conditions including at least one offailure or success of the first predetermined malicious attack. Forexample, the result data analyzer 117 can be configured to producegraphical output corresponding to the result data (e.g., failure orsuccess of a simulated attack) including a pie chart (see FIG. 6), acolor indication for each attack stage (see FIG. 6), a pie chart byattack types (see FIG. 7), an indication of number of attacks thatsucceeded or fails for each attack stage (see FIG. 7), or a heat map(see FIG. 8).

E. Packet Capture (PCAP) Traffic Recording

Packet capture (PCAP) files can be used to facilitate testing thesecurity posture of a production network. In general, PCAP files may befiles containing a history of network traffic sent between two or morepairs of computing devices during the execution of an actual attack orother malicious behavior. PCAP files may be generated, for example, by anetwork analyzer application that stores a record of the network trafficbetween the two or more devices while the devices are transmittingnetwork packets that correspond to the malicious behavior. By capturingsuch malicious behavior in the form of a PCAP file, the actual networktraffic associated with the behavior can be analyzed and replicated forthe purpose of testing how a production network responds to a similarattack. The security of the network can then analyzed from differentperspectives. For example, an entire network path can be tested based ona PCAP file that corresponds to an attack conducted along a networkpath, or an endpoint of a path can be tested in isolation, for examplebased on a PCAP file that corresponds to an attack conducted only on anetwork endpoint. In some implementations, a server can receive theresults of the various attacks (e.g., results determined by each of aplurality of nodes involved in an attack). The server can then aggregatethe results and provide the aggregated results to a networkadministrator. In some implementations, the server can provide theresults to an event console, which may be a third party hardware deviceor software application having an API that allows the server tocommunicate with the event console.

FIG. 12 is a flowchart of an example method 1200 for controlledexecution of a malicious behavior between multiple different componentsin a production network to test at least a portion of a security systemof the production network. In brief overview, the method 1200 includesidentifying, by a server, a packet capture (PCAP) file to use forproviding instructions for controlled execution of malicious behavior ona plurality of different components of a production network (step 1210).The method 1200 includes identifying, by the server, a first attack toexecute against a first device of a plurality of different components, asecond attack to execute against a second device of the plurality ofdifferent components, and a third attack to execute against a networkcomponent of the plurality of different components intermediary to thefirst end point device and the second device (step 1220). The method1200 includes communicating, by the server based on at least the PCAPfile, a first set of instructions to a first pair of attack and targetnodes in the production network to initiate the malicious behavior ofthe PCAP file against the first device, a second set of instructions toa second pair of attack and target nodes in the production network toinitiate the malicious behavior of the PCAP file against the seconddevice, and a third set of instructions to a third pair of attack andtarget nodes in the production network to initiate the maliciousbehavior of the PCAP file against the network component intermediary tothe first device and the second device (step 1230). The method 1200 alsoincludes aggregating, by the server, results of each of the firstattack, the second attack, and the third attack to provide an aggregatedview of the malicious behavior between the first device and the seconddevice via the network component (step 1240).

Referring again to FIG. 12, and in greater detail, the method 1200includes identifying, by a server, a packet capture (PCAP) file to usefor providing instructions for controlled execution of maliciousbehavior on a plurality of different components of a production network(step 1210). In some implementations, the server can be a computingdevice such as the planner 115 shown in FIGS. 1A and 1B. Thus, in someimplementations, the server can receive a plurality of PCAP files, whichmay be stored, for example, in a database of the planner such as thedatabase 118 shown in FIG. 1B. In general, a PCAP file can include arecord of the data packets transmitted between a pair of devices duringan attack corresponding to malicious behavior. In some implementations,the particular PCAP file identified in step 1210 can be selected basedon a particular type of attack or malicious behavior that is to betested in the production network.

In some implementations, the server can analyze a PCAP file to extractan application-layer record of requests and responses exchanged betweenthe attacker and the target in the network traffic represented by thePCAP file. The server also may extract an actual malware file from thePCAP file. The PCAP file will typically contain low-level packetinformation related to data exchanged during the attack. However, theserver can process the PCAP file to extract the higher-level applicationlayer record of each request and response, to create a set ofinstructions for network nodes to accurately replicate the attack withinthe production network. In some implementations, the server can beconfigured to process a PCAP file by first identifying each hostconversation within the PCAP file. For example, the server can make thisdetermination based on the communication protocol used in the PCAP file.In some implementations, the server also can determine the type ofapplication traffic represented by the PCAP file, such as HTTP trafficthat may be sent using TCP, or DNS traffic that may be sent using UDP.In some implementations, the determination of the type of applicationtraffic represented by the PCAP file can be made based on the use ofapplication signatures in the PCAP file.

The method 1200 includes identifying, by the server, a first attack toexecute against a first device of a plurality of different components, asecond attack to execute against a second device of the plurality ofdifferent components, and a third attack to execute against a networkcomponent of the plurality of different components intermediary to thefirst end point device and the second device (step 1220). In someimplementations, identifying the sequence of first, second, and thirdattacks can allow for more complete testing of one or more network pathsin the production network. As illustrated in the enterprise network 105shown in FIG. 1A, network paths may be complex and may traverse multipledevices within the network. For example, there is no direct path betweenthe computing device 120 a in the workstations zone of the enterprisenetwork 105 and the computing device 130 a in the internal servers zoneof the enterprise network 105. Thus, to test security along this path,multiple computing devices (i.e., at least the computing device 120 a,the IPSs 155 a and 155 c, and the router 145) must be involved, as theyare positioned along the path joining the computing device 120 and thecomputing device 130 a.

In some implementations, the server can be configured to select thesequence of the first attack, the second attack, and the third attackbased on a particular network path under test. IN some implementations,at least one of the first device and the second device can be anendpoint device. Referring again to FIG. 1A, for example, the firstdevice targeted for attack could be the computing device 120 a, thesecond device targeted for the second attack could be the router 145,and the network component targeted for the third attack could be the IPS155 a, which is intermediary to both the first device (i.e., thecomputing device 120 a) and the second device (i.e., the router 145). Itshould be understood that these devices are illustrative only, and thatin practice, the server can be configured to identify any set ofcomputing devices in the production network as the first device, thesecond device, and the network component, depending on the desirednetwork path that is to be subjected to the test. For example, in someimplementations, the server can be configured to target the controllers140 a-140 e shown in FIG. 1A, rather than the production equipment, suchas the computing devices 120, 125, 130, and 135, in order to protect theproduction equipment from malicious behavior.

The method 1200 includes communicating, by the server based on at leastthe PCAP file, a first set of instructions to a first pair of attack andtarget nodes in the production network to initiate the maliciousbehavior of the PCAP file against the first device, a second set ofinstructions to a second pair of attack and target nodes in theproduction network to initiate the malicious behavior of the PCAP fileagainst the second device, and a third set of instructions to a thirdpair of attack and target nodes in the production network to initiatethe malicious behavior of the PCAP file against the network componentintermediary to the first device and the second device (step 1230). Asdescribed above, the server can process the PCAP file to determine thenetwork traffic between each pair of network devices represented in thePCAP file. Based on the determined network traffic, the server cangenerate a set of instructions that will cause the pairs of devices tosend network traffic to replicate the attack represented in the PCAPfile. The instructions can include any type and form of data relating tothe attacks to be carried out. For example, the instructions can includea sequence of network packets that should be sent between each pair ofdevices for each respective attack. The instructions also can includemetadata that can facilitate execution of the malicious behavior. Theserver can determined such metadata based on the PCAP file. The metadatacan be any form data indicating the types and/or content of one or morerequests or responses that each device should send or receive. In someimplementations, the instructions also can include timing or otherconditional information for the attacks. For example, the instructionsmay cause the second attack to be carried out only upon completion ofthe first attack.

The method 1200 also includes aggregating, by the server, results ofeach of the first attack, the second attack, and the third attack toprovide an aggregated view of the malicious behavior between the firstdevice and the second device via the network component (step 1240).After the server communicates the first, second, and third sets ofinstructions to the respective pairs of attack and target nodes in step1230, the pairs of attack and target nodes can perform the attacksaccording to the instructions. Generally, execution of the attacksincludes sending and receiving network traffic between the respectivepairs of nodes according to the instructions. In some implementations,the pairs of nodes can determine results of their respective attacks.For example, as described above, the instructions that the servercommunicates to each node can include data (or metadata) indicatingresponses that each node should expect to receive during the executionof the attack. The nodes can compare the actual responses they receiveto the expected responses to determine a match, which may indicate asuccessful attack, or a mismatch, which may indicate an unsuccessfulattack. After determining the results of their respective attacks, eachattack-target node pair can transmit the results back to the server. Theserver can aggregate the results it receives from the attack-target nodepairs, and can provide the aggregated results to an administrator with acomplete description of how the production network responded to theattack. In some implementations, the server can provide the aggregatedresults to a third party event console, such as the SIEM 160 shown inFIG. 1A. For example, the server can communicate with the event consolevia an API provided by the event console. In some implementations, theserver also can request event information from the event console. Forexample, the server can receive an indication of events detected by theevent console during any of the first, second, or third attacks, and cancorrelate the events to the results of the attacks it receives from thenodes involved in the attacks.

F. Computing Devices

FIGS. 13A and 13B are block diagrams of implementations of an examplecomputing device 1300. In some implementations, the computing device1300 may be useful for implementing the planner 115 or the controller140 shown in FIGS. 1A-1C. As shown in FIGS. 13A and 13B, each computingdevice 1300 includes a central processing unit 1301, and a main memoryunit 1322. As shown in FIG. 1E, a computing device 1300 may include avisual display device 1324, a keyboard 1326 and/or a pointing device1327, such as a mouse. Each computing device 1300 may also includeadditional optional elements, such as one or more input/output devices1330 a-1330 b (generally referred to using reference numeral 1330), anda cache memory 1340 in communication with the central processing unit1301.

The central processing unit 1301 is any logic circuitry that responds toand processes instructions fetched from the main memory unit 1322. Inmany embodiments, the central processing unit is provided by amicroprocessor unit, such as: those manufactured by Intel Corporation ofMountain View, Calif.; those manufactured by Motorola Corporation ofSchaumburg, Ill.; those manufactured by Transmeta Corporation of SantaClara, Calif.; the RS/6000 processor, those manufactured byInternational Business Machines of White Plains, N.Y.; or thosemanufactured by Advanced Micro Devices of Sunnyvale, Calif. Thecomputing device 1300 may be based on any of these processors, or anyother processor capable of operating as described herein.

Main memory unit 1322 may be one or more memory chips capable of storingdata and allowing any storage location to be directly accessed by thecentral processing unit 1301, such as Static random access memory(SRAM), Burst SRAM or SynchBurst SRAM (BSRAM), Dynamic random accessmemory (DRAM), Fast Page Mode DRAM (FPM DRAM), Enhanced DRAM (EDRAM),Extended Data Output RAM (EDO RAM), Extended Data Output DRAM (EDODRAM), Burst Extended Data Output DRAM (BEDO DRAM), Enhanced DRAM(EDRAM), synchronous DRAM (SDRAM), JEDEC SRAM, PC100 SDRAM, Double DataRate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM),Direct Rambus DRAM (DRDRAM), or Ferroelectric RAM (FRAM). The mainmemory 1322 may be based on any of the above described memory chips, orany other available memory chips capable of operating as describedherein. In the embodiment shown in FIG. 13A, the central processing unit1301 communicates with main memory 1322 via a system bus 1350 (describedin more detail below). FIG. 13B depicts an embodiment of a computingdevice 1300 in which the processor communicates directly with mainmemory 1322 via a memory port 1303. For example, in FIG. 13B the mainmemory 1322 may be DRDRAM.

FIG. 13B depicts an embodiment in which the central processing unit 1301communicates directly with cache memory 1340 via a secondary bus,sometimes referred to as a backside bus. In other embodiments, thecentral processing unit 1301 communicates with cache memory 1340 usingthe system bus 1350. Cache memory 1340 typically has a faster responsetime than main memory 1322 and is typically provided by SRAM, BSRAM, orEDRAM. In the embodiment shown in FIG. 13B, the central processing unit1301 communicates with various I/O devices 1330 via a local system bus1350. Various busses may be used to connect the central processing unit1301 to any of the I/O devices 1330, including a VESA VL bus, an ISAbus, an EISA bus, a MicroChannel Architecture (MCA) bus, a PCI bus, aPCI-X bus, a PCI-Express bus, or a NuBus. For embodiments in which theI/O device is a video display 1324, the central processing unit 1301 mayuse an Advanced Graphics Port (AGP) to communicate with the display1324. FIG. 13B depicts an embodiment of a computer 1300 in which thecentral processing unit 1301 communicates directly with I/O device 1330b via HyperTransport, Rapid I/O, or InfiniBand. FIG. 13B also depicts anembodiment in which local busses and direct communication are mixed: thecentral processing unit 1301 communicates with I/O device 1330 b using alocal interconnect bus while communicating with I/O device 1330 adirectly.

The computing device 1300 may support any suitable installation device1316, such as a floppy disk drive for receiving floppy disks such as3.5-inch, 5.25-inch disks or ZIP disks, a CD-ROM drive, a CD-R/RW drive,a DVD-ROM drive, tape drives of various formats, USB device, hard-driveor any other device suitable for installing software and programs suchas any program related to the planner 115 or the controller 140. Thecomputing device 1300 may further comprise a storage device 1328, suchas one or more hard disk drives or redundant arrays of independentdisks, for storing an operating system and other related software, andfor storing application software programs such as any program related toeither the planner 115 or the controller 140. Optionally, any of theinstallation devices 1316 could also be used as the storage device 1328.Additionally, the operating system and the software can be run from abootable medium, for example, a bootable CD, such as KNOPPIX®, abootable CD for GNU/Linux that is available as a GNU/Linux distributionfrom knoppix.net.

Furthermore, the computing device 1300 may include a network interface1318 to interface to a Local Area Network (LAN), Wide Area Network (WAN)or the Internet through a variety of connections including, but notlimited to, standard telephone lines, LAN or WAN links (e.g., 802.11,T1, T3, 56 kb, X.25), broadband connections (e.g., ISDN, Frame Relay,ATM), wireless connections, or some combination of any or all of theabove. The network interface 1318 may comprise a built-in networkadapter, network interface card, PCMCIA network card, card bus networkadapter, wireless network adapter, USB network adapter, modem or anyother device suitable for interfacing the computing device 1300 to anytype of network capable of communication and performing the operationsdescribed herein.

A wide variety of I/O devices 1330 a-1330 n may be present in thecomputing device 1300. Input devices include keyboards, mice, trackpads,trackballs, microphones, and drawing tablets. Output devices includevideo displays, speakers, inkjet printers, laser printers, anddye-sublimation printers. The I/O devices 1330 may be controlled by anI/O controller 1323 as shown in FIG. 13A. The I/O controller may controlone or more I/O devices such as a keyboard 1326 and a pointing device1327, e.g., a mouse or optical pen. Furthermore, an I/O device may alsoprovide storage 1328 and/or an installation medium 1316 for thecomputing device 1300. In still other embodiments, the computing device1300 may provide USB connections to receive handheld USB storage devicessuch as the USB Flash Drive line of devices manufactured by TwintechIndustry, Inc. of Los Alamitos, Calif.

In some embodiments, the computing device 1300 may comprise or beconnected to multiple display devices 1324 a-1324 n, which each may beof the same or different type and/or form. As such, any of the I/Odevices 1330 a-1330 n and/or the I/O controller 1323 may comprise anytype and/or form of suitable hardware, software, or combination ofhardware and software to support, enable or provide for the connectionand use of multiple display devices 1324 a-1324 n by the computingdevice 1300. For example, the computing device 1300 may include any typeand/or form of video adapter, video card, driver, and/or library tointerface, communicate, connect or otherwise use the display devices1324 a-1324 n. In one embodiment, a video adapter may comprise multipleconnectors to interface to multiple display devices 1324 a-1324 n. Inother embodiments, the computing device 1300 may include multiple videoadapters, with each video adapter connected to one or more of thedisplay devices 1324 a-1324 n. In some embodiments, any portion of theoperating system of the computing device 1300 may be configured forusing multiple displays 1324 a-1324 n. In other embodiments, one or moreof the display devices 1324 a-1324 n may be provided by one or moreother computing devices, such as computing devices 1300 a and 1300 bconnected to the computing device 1300, for example, via a network.These embodiments may include any type of software designed andconstructed to use another computer's display device as a second displaydevice 1324 a for the computing device 1300. One ordinarily skilled inthe art will recognize and appreciate the various ways and embodimentsthat a computing device 1300 may be configured to have multiple displaydevices 1324 a-1324 n.

In further embodiments, an I/O device 1330 may be a bridge 1370 betweenthe system bus 1350 and an external communication bus, such as a USBbus, an Apple Desktop Bus, an RS-232 serial connection, a SCSI bus, aFireWire bus, a FireWire 800 bus, an Ethernet bus, an AppleTalk bus, aGigabit Ethernet bus, an Asynchronous Transfer Mode bus, a HIPPI bus, aSuper HIPPI bus, a SerialPlus bus, a SCI/LAMP bus, a FibreChannel bus,or a Serial Attached small computer system interface bus.

A computing device 1300 of the sort depicted in FIGS. 13A and 13Btypically operate under the control of operating systems, which controlscheduling of tasks and access to system resources. The computing device1300 can be running any operating system such as any of the versions ofthe Microsoft® Windows operating systems, the different releases of theUnix and Linux operating systems, any version of the Mac OS® forMacintosh computers, any embedded operating system, any real-timeoperating system, any open source operating system, any proprietaryoperating system, any operating systems for mobile computing devices, orany other operating system capable of running on the computing deviceand performing the operations described herein. Typical operatingsystems include: WINDOWS 3.x, WINDOWS 95, WINDOWS 98, WINDOWS 2000,WINDOWS NT 3.51, WINDOWS NT 4.0, WINDOWS CE, and WINDOWS XP, all ofwhich are manufactured by Microsoft Corporation of Redmond, Wash.;MacOS, manufactured by Apple Computer of Cupertino, Calif.; OS/2,manufactured by International Business Machines of Armonk, N.Y.; andLinux, a freely-available operating system distributed by Caldera Corp.of Salt Lake City, Utah, or any type and/or form of a UNIX operatingsystem, among others.

In other embodiments, the computing device 1300 may have differentprocessors, operating systems, and input devices consistent with thedevice. For example, in one embodiment the computer 1300 is a Treo 180,270, 1060, 600 or 650 smart phone manufactured by Palm, Inc. In thisembodiment, the Treo smart phone is operated under the control of thePalmOS operating system and includes a stylus input device as well as afive-way navigator device. Moreover, the computing device 1300 can beany workstation, desktop computer, laptop or notebook computer, server,handheld computer, mobile telephone, any other computer, or other formof computing or telecommunications device that is capable ofcommunication and that has sufficient processor power and memorycapacity to perform the operations described herein.

It should be understood that the systems described above may providemultiple ones of any or each of those components and these componentsmay be provided on either a standalone machine or, in some embodiments,on multiple machines in a distributed system. The systems and methodsdescribed above may be implemented as a method, apparatus or article ofmanufacture using programming and/or engineering techniques to producesoftware, firmware, hardware, or any combination thereof. In addition,the systems and methods described above may be provided as one or morecomputer-readable programs embodied on or in one or more articles ofmanufacture. The term “article of manufacture” as used herein isintended to encompass code or logic accessible from and embedded in oneor more computer-readable devices, firmware, programmable logic, memorydevices (e.g., EEPROMs, ROMs, PROMs, RAMs, SRAMs, etc.), hardware (e.g.,integrated circuit chip, Field Programmable Gate Array (FPGA),Application Specific Integrated Circuit (ASIC), etc.), electronicdevices, a computer readable non-volatile storage unit (e.g., CD-ROM,floppy disk, hard disk drive, etc.). The article of manufacture may beaccessible from a file server providing access to the computer-readableprograms via a network transmission line, wireless transmission media,signals propagating through space, radio waves, infrared signals, etc.The article of manufacture may be a flash memory card or a magnetictape. The article of manufacture includes hardware logic as well assoftware or programmable code embedded in a computer readable mediumthat is executed by a processor. In general, the computer-readableprograms may be implemented in any programming language, such as LISP,PERL, C, C++, C#, PROLOG, or in any byte code language such as JAVA. Thesoftware programs may be stored on or in one or more articles ofmanufacture as object code.

While various embodiments of the methods and systems have beendescribed, these embodiments are exemplary and in no way limit the scopeof the described methods or systems. Those having skill in the relevantart can effect changes to form and details of the described methods andsystems without departing from the broadest scope of the describedmethods and systems. Thus, the scope of the methods and systemsdescribed herein should not be limited by any of the exemplaryembodiments and should be defined in accordance with the accompanyingclaims and their equivalents.

What is claimed is:
 1. A method for configuring a controlled executionof a sequence of attacks in a production network to test a securitysystem, the method comprising: (a) receiving, via a planning tool,selection of a first attack node and a second attack node viainteraction with a graphical representation of a topology of aproduction network, a first specification of a first attack of aplurality of attacks in an attack sequence, the first specificationidentifying the first attack node and a first target node in aproduction network, a first network path selected between the firstattack node and the first target node via interaction with a graphicalrepresentation of links between nodes in the production network, and afirst predetermined malicious behavior; (b) receiving, via the planningtool, a second specification of a second attack of the plurality ofattack in the attack sequence, the second specification identifying thesecond attack node and a second target node in the production network, asecond network path selected between the second attack node and thesecond target node, and a second predetermined malicious behaviordifferent than the first predetermined malicious behavior; (c)receiving, via the planning tool, identification of one or moreconditions upon which to execute the second attack subsequent toexecution of the first attack in the attack sequence, wherein the one ormore conditions comprises one of failure or success of the first attack;(d) storing, by the planning tool, the attack sequence comprising thefirst specification of the first attack and the second specification ofthe second attack; (e) identifying, by the planning tool, the storedattack sequence for controlled execution within the production networkto test at least a portion of a security system.
 2. The method of claim1, wherein (a) further comprises receiving a selection of a pairing ofthe first attack node and the second attack node via interaction with agraphical representation of a topology of the production network.
 3. Themethod of claim 1, wherein (a) further comprises receiving the selectionof the first network path via interaction with the graphicalrepresentation of links between nodes in the production network.
 4. Themethod of claim 1, wherein (b) further comprises receiving the secondspecification identifying the second predetermined malicious behaviordifferent than the first predetermined malicious behavior.
 5. The methodof claim 1, wherein (b) further comprises receiving the secondspecification identifying the second attack node and the second targetnode comprising types of operating systems different than at least oneof the first attack node or the first target node.
 6. The method ofclaim 1, wherein (c) further comprises identifying the one or moreconditions as a time duration between the first attack and the secondattack.
 7. The method of claim 1, wherein (c) further comprisesidentifying the one or more conditions as detection of one of failure ofor success of the first attack.
 8. A system for configuring a controlledexecution of a sequence of attacks in a production network to test asecurity system, the system comprising: a planning tool executable on aprocess coupled to memory and configured to a selection of a firstattack node and a second attack node via interaction with a graphicalrepresentation of a topology of a production network receive a firstspecification of a first attack of a plurality of attacks in an attacksequence, the first specification identifying the first attack node anda first target node in a production network, a first network pathselected between the first attack node and the first target node viainteraction with a graphical representation of links between nodes inthe production network, and a first predetermined malicious behavior;receive a second specification of a second attack of the plurality ofattack in the attack sequence, the second specification identifying thesecond attack node and a second target node in the production network, asecond network path selected between the second attack node and thesecond target node, and a second predetermined malicious behaviordifferent than the first predetermined malicious behavior; receiveidentification of one or more conditions upon which to execute thesecond attack subsequent to execution of the first attack in the attacksequence; store the attack sequence comprising the first specificationof the first attack and the second specification of the second attack;identify the stored attack sequence for controlled execution within theproduction network to test at least a portion of a security system; andwherein the one or more conditions comprises one of failure or successof the first attack.
 9. The system of claim 8, wherein the planner isfurther configured to receive a selection of a pairing of the firstattack node and the second attack node via interaction with a graphicalrepresentation of a topology of the production network.
 10. The systemof claim 8, wherein the planning tool is further configured to receivethe selection of the first network path via interaction with thegraphical representation of links between nodes in the productionnetwork.
 11. The system of claim 8, wherein the planner is furtherconfigured to receive the second specification identifying the secondpredetermined malicious behavior different than the first predeterminedmalicious behavior.
 12. The system of claim 8, wherein the planner isfurther configured to receive the second specification identifying thesecond attack node and the second target node comprising types ofoperating systems different than at least one of the first attack nodeor the first target node.
 13. The system of claim 8, wherein the planneris further configured to receive identification of the one or moreconditions as a time duration between the first attack and the secondattack.
 14. The system of claim 8, wherein the planner is furtherconfigured to receive identification of the one or more conditions asdetection of one of failure of or success of the first attack.
 15. Amethod for controlled execution of a sequence of attacks in a productionnetwork to test at least a portion of a security system of theproduction network, the method comprising: (a) identifying, by amanager, an attack sequence comprising a first attack and a secondattack and one or more conditions upon which to execute the secondattack subsequent to the first attack, wherein the first attackspecifies a first attack node and a first target node in a productionnetwork, a first network path selected between the first attack node andthe first target node via interaction with a graphical representation oflinks between nodes in the production network, and a first predeterminedmalicious behavior and wherein the second attack specifies a secondattack node and a second target node in the production network, a secondnetwork path selected between the second attack node and the secondtarget node, and a second predetermined malicious behavior differentthan the first predetermined malicious behavior, wherein the firstattack node and the second attack node selected via interaction with agraphical representation of a topology of a production network; (b)communicating, by the manager, instructions to the first attack node andthe first target node in the production network to initiate the firstpredetermined malicious behavior via the first path selected between thefirst attack node and the first target node; (c) monitoring, by amonitor, execution of the first attack between the first attack node andthe first target node for the one or more conditions, wherein the one ormore conditions comprises one of failure or success of the first attack;(d) determining, by the manager responsive to detecting the one or moreconditions via monitoring, to execute the second attack of the attacksequence.
 16. The method of claim 15, further comprising communicating,by the simulation data manager, instructions to the second attack nodeand the second target node in the production network to initiate thesecond predetermined malicious behavior via the second path selectedbetween the second attack node and the second target node.
 17. Themethod of claim 15, wherein (a) further comprises identifying, by thesimulation data manager, the attack sequence comprising the first attacknode and the first target node on nodes with a type of operating systemdifferent than at least one of the second attack node and second targetnode.
 18. The method of claim 15, wherein (a) further comprisesidentifying, by the simulation data manager, the attack sequencecomprising the first predetermined malicious behavior different than thesecond predetermined malicious behavior.
 19. The method of claim 15,wherein (a) further comprises identifying, by the simulation datamanager, the attack sequence comprising the first selected network pathdifferent than the second selected network path.
 20. The method of claim15, wherein (b) further comprises transmitting, responsive toinstructions from the planner by the first attack node via a connectionto the first target node and via at least a portion of the securitysystem of the production network, network traffic comprising the firstpredetermined malicious behavior.
 21. The method of claim 15, wherein(c) further comprises detecting via monitoring the one or moreconditions comprising a time duration.
 22. The method of claim 15,wherein (c) further comprises detecting via monitoring the one or moreconditions comprising at least one of failure or success of the firstpredetermined malicious attack.
 23. A system for controlled execution ofa sequence of attacks in a production network to test at least a portionof a security system of the production network, the system comprising: amanager executable on a processor coupled to memory and configured to:identify an attack sequence comprising a first attack and a secondattack and one or more conditions upon which to execute the secondattack subsequent to the first attack, wherein the first attackspecifies a first attack node and a first target node in a productionnetwork, a first network path selected between the first attack node andthe first target node via interaction with a graphical representation oflinks between nodes in the production network, and a first predeterminedmalicious behavior and wherein the second attack specifies a secondattack node and a second target node in the production network, a secondnetwork path selected between the second attack node and the secondtarget node, and a second predetermined malicious behavior differentthan the first predetermined malicious behavior; communicateinstructions to the first attack node and the first target node in theproduction network to initiate the first predetermined maliciousbehavior via the first path selected between the first attack node andthe first target node; a monitor configured to monitor execution of thefirst attack between the first attack node and the first target node forthe one or more conditions, wherein the one or more conditions comprisesone of failure or success of the first attack; wherein the first attacknode and the second attack node selected via interaction with agraphical representation of a topology of a production network; andwherein the manager, responsive to the monitor detecting the one or moreconditions, is configured to determine to execute the second attack ofthe attack sequence.
 24. The system of claim 23, wherein the simulationdata manager is further configured to communicate instructions to thesecond attack node and the second target node in the production networkto initiate the second predetermined malicious behavior via the secondpath selected between the second attack node and the first second node.25. The system of claim 23, wherein the simulation data manager isfurther configured to identify the attack sequence comprising the firstattack node and the first target node on nodes with a type of operatingsystem different than at least one of the second attack node and secondtarget node.
 26. The system of claim 23, wherein the simulation datamanager is further configured to identify the attack sequence comprisingthe first predetermined malicious behavior different than the secondpredetermined malicious behavior.
 27. The system of claim 23, whereinthe simulation data manager is further configured to identify the attacksequence comprising the first selected network path different than thesecond selected network path.
 28. The system of claim 23, wherein thefirst attack node is configured to, responsive to instructions from aplanner, transmit via a connection to the first target node and via atleast a portion of the security system of the production network,network traffic comprising the first predetermined malicious behavior.29. The system of claim 23, wherein the result data analyzer is furtherconfigured to detect the one or conditions comprising a time duration.30. The system of claim 23, wherein the result data analyzer is furtherconfigured to detect the one or conditions comprising at least one offailure or success of the first predetermined malicious attack.